I see even here on DEV many people struggle after school/college/university to get their first job in IT as web-designer / web-developer or something related. It may be bit hard, as there are many "wanna be devs" and many skills are needed for efficient work even as junior.

But there is important alternative!

I often recommend to start career not as dev, but as "quality assurance" specialist, or "software tester". It's very good to get introduced to dev team work and processes and may require less technical skills but good wits!

And as for your programming talents - attempt to apply for qa automation (not completely manual testing).

If you never heard of it - let me briefly show how it looks for web testing!

Typical Problem

Here is a Fake Social Network - you may read more about it in this exercise. Each page represents some profile. Profiles are linked via friends and wall links. Try clicking them to see this...

Now suppose you, as QA specialist, are asked to

Crawl through many (all) pages you can reach from this starting one - and make sure all links are not broken. I.e. no link should lead to 404 page or something like this.

How do we do this? While for scrapping we use slightly different tools, for web-testing we use

Selenium!

It's great tool which programmatically manages your browser. Let's see it in action. You need just two things to use it from Python:

• install "binding" library for it, e.g. pip install selenium (I do this in virtualenv usually)
• download Selenium driver for your chosen browser, e.g. for Chrome here - and put it, say, in the current directory along with your future python code.

Now the code. It's quite simple. First, import Selenium-related stuff:

from selenium.webdriver import Chrome
from selenium.webdriver.chrome.options import Options

Then open browser and open the page in it:

opts = Options()
#opts.set_headless()
browser = Chrome(options=opts)

browser.get('http://codeabbey.github.io/social-network/')

Save this as social-test.py and run with python3 social-test.py. If everything is good, you should see the browser opens.

If python complains it can't find seleniumdriver file, check its name and (for linux) has current directory in the PATH. Maybe add it, e.g. export HOME=$HOME:.

Now let's close the new browser window and try doing something clever by adding more lines to file:

friends = browser.find_elements_by_css_selector(".friends a")
for f in friends:
    print(f.get_attribute('href'))

browser.close()

Obviously first line gets all link DOM element in Friends pane. Then we iterate through them and print their href attribute, so you should see

http://codeabbey.github.io/social-network/jessica-h-klein.html
http://codeabbey.github.io/social-network/dan-z-wagner.html
http://codeabbey.github.io/social-network/dave-b-johnson.html
http://codeabbey.github.io/social-network/paulina-y-cleveland.html

After which browser closes (as we put command for such action).

I think this is enough for introduction. Try to extend the code, so it follows links (either with .click() call or by directly opening them).

As premium exercise try to discover all the pages there, without reopening the same many times (should be about 1000). Perhaps, solve the initial task on scraping or write code for discovering shortest relation chain.

Feel free to refer to Python Selenium documentation or just search web and stackoverflow for hints. The documentation is not ideal, but you'll find zounds of examples.

And as for "why learn it" - just type in "Selenium" in your favorite job-searching site. E.g. for monster it looks like this

Good luck! Thanks for reading that far. Hopefully it was not completely useless.