Some websites can block access to prevent web scraping, that can be easily detected if your Python script is sending multiple requests in a short period of time.
To not get banned you can try to add random delays between queries.
For this you can use the Python’s sleep() function that suspends (waits) execution of the current thread for a given number of seconds with the randint() function that returns a random integer.
Cool Tip: How to set the ‘User-Agent’ HTTP request header in Python! Read More →
Random Sleep Timer in Python
Use the following Python’s code snippet to sleep a random number of seconds:
from random import randint
from time import sleep
# Sleep a random number of seconds (between 1 and 5)
sleep(randint(1,5))
Most of the time you need to create some random delays between iterations in a loop.
In the example below the Python is sleeping a random number of seconds between printing a message:
from random import randint
from time import sleep
while True:
delay = randint(1,5)
print("Sleep " + str(delay) + "s")
sleep(delay)
Sample output:
Sleep 5s Sleep 1s Sleep 2s Sleep 2s Sleep 4s