Question

I need help with an assignment, it invovles techniques such as web scraping to retrieve data from the Internet using both Python built-in functions as well as third-party libraries. You can use Python's default editor IDLE (Integrated Development and Learning Environment) or a Python editor of your choice to complete this assignment. I need to do the  following:

Write a program to test if a given page is found or not on the server.

Write a Python program to download and display the content of robots.txt for http://www.capella.edu.

Install the third-party library Beautiful Soup and review the Beautiful Soup Documentation,

Use Beautiful Soup to write a Python program that will extract all of the HTML links from http://www.capella.edu.

Test your code and document testing by taking screenshots.

Answer 1

#source code:

#output:

#if you have any doubts comment below....