elenium is a powerful tool for controlling web browsers through programs and performing browser automation. It is particularly useful for testing web applications and is also used for automating repetitive web-based administration tasks or web scraping. When used with Python, Selenium allows you to automate browser actions like clicking buttons, filling out forms, and fetching web page data with ease.

Getting Started with Selenium in Python

  1. Installation

First, you need to install the Selenium package. This can be done using pip, Python’s package installer. Make sure you have Python and pip installed on your system before you proceed.

pip install selenium
  1. Web Drivers

Selenium requires a driver to interface with the chosen browser. Chrome, for instance, requires chromedriver, which needs to be installed before you can control the browser with Selenium. Drivers for other browsers like Firefox (geckodriver) and Edge are also available.

Download the driver for your browser from the appropriate websites:

  • Chrome
  • Firefox
  • Edge Ensure the driver is in your PATH, or you can specify the location directly in your code.

In the past you had to install the drivers manually. With Selenium 4 and newer versions, you can simplify the management of driver executables like chromedriver for Chrome, geckodriver for Firefox, etc., by using the webdriver_manager package. This package automatically checks the version of the browser installed on your system and downloads the appropriate driver if it’s not present or if it needs an update. This can save you the hassle of manually downloading and updating these drivers.

Installation

First, you need to also install the webdriver_manager package. You can do this using pip:

pip install selenium webdriver_manager

Example Code

Here’s how to use Selenium with webdriver_manager to open a web page in Chrome:


from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager

# Setup Chrome with webdriver_manager
driver = webdriver.Chrome(ChromeDriverManager().install())
# driver = webdriver.Firefox(executable_path=GeckoDriverManager().install())
# driver = webdriver.Edge(EdgeChromiumDriverManager().install())

# Open a webpage
driver.get("https://www.example.com")

# Fetch the title of the webpage
print(driver.title)  # Prints the title of the page

# Close the browser
driver.quit()

  1. WebDriver

The WebDriver is the key interface that drives the browser. It sends commands to the browser and retrieves results. Different browsers will have different implementations of the WebDriver.

  1. WebElement

The WebElement represents an HTML element. WebElements can be found by using driver methods like find_element_by_* - for example, find_element_by_id, find_element_by_name, find_element_by_xpath, etc.

  1. Interacting with the Page

Once you have WebElement instances, you can do things like click on them, input text, read the data from them, or even execute JavaScript through them.

  1. Waits

Selenium provides two types of waits - implicit and explicit, to handle scenarios where elements take some time to become available or visible after page load:

Implicit Wait: Makes WebDriver poll the DOM for a certain amount of time when trying to find any element.

driver.implicitly_wait(10)  # seconds


Explicit Wait: Makes WebDriver wait for a certain condition to occur before proceeding further with execution.

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

element = WebDriverWait(driver, 10).until(
    EC.presence_of_element_located((By.ID, "myElement"))
)

Best Practices

Always use explicit waits rather than implicit waits to handle synchronization.

Organize your Selenium code using Page Object Models (POM) for maintainability and readability.

Clean up your resources by closing browsers with driver.quit() after your script runs