silver imac on desk
Computer ready to run python asynchronously

This article details how to use Python asynchronously. Out of the box, Python is a synchronous language but the asyncio library allows users to run code concurrently. We can accomplish this by using await/async keyword syntax. This example uses a simple async API call loop.

The Problem

Let’s say we’ve been using the Python requests library to get text data from https://GeeksforGeeks.com. However, our program is taking longer to extract all the data because of the time required to process each request. Here is an example of a basic request loop to a list of endpoints.

import requests
import time

def getGFG():
    a = time.perf_counter()
    d = {}
    endpoints = ['https://www.geeksforgeeks.org/', 'https://www.geeksforgeeks.org/python-programming-language/?ref=shm', 'https://www.geeksforgeeks.org/fundamentals-of-algorithms/?ref=shm','https://www.geeksforgeeks.org/', 'https://www.geeksforgeeks.org/python-programming-language/?ref=shm', 'https://www.geeksforgeeks.org/fundamentals-of-algorithms/?ref=shm','https://www.geeksforgeeks.org/', 'https://www.geeksforgeeks.org/python-programming-language/?ref=shm', 'https://www.geeksforgeeks.org/fundamentals-of-algorithms/?ref=shm','https://www.geeksforgeeks.org/', 'https://www.geeksforgeeks.org/python-programming-language/?ref=shm', 'https://www.geeksforgeeks.org/fundamentals-of-algorithms/?ref=shm']
    for i in endpoints:
        text = requests.get(i).text
        d[text] = text
    b = time.perf_counter()
    print(b - a, 'seconds')
cgetGFG()

The above code works well, but it runs fairly slowly (~5 seconds for 3 requests). Here, we simply request a set of urls from a list and append them into a dictionary. Instead of the script running Python requests asynchronously, the requests happen on an execute, wait, execute basis. We can speed up these requests with a few simple changes by utilizing the asyncio and aiohttp Python libraries.

import aiohttp
import asyncio

async def get_GFG_task(session, url):
    async with session.get(url) as resp:
        text = await resp.text()
        return text

async def getGFGAsync():
    d = {}
    a = time.perf_counter()
    async with aiohttp.ClientSession() as session:

        tasks = []

        endpoints = ['https://www.geeksforgeeks.org/', 'https://www.geeksforgeeks.org/python-programming-language/?ref=shm', 'https://www.geeksforgeeks.org/fundamentals-of-algorithms/?ref=shm','https://www.geeksforgeeks.org/', 'https://www.geeksforgeeks.org/python-programming-language/?ref=shm', 'https://www.geeksforgeeks.org/fundamentals-of-algorithms/?ref=shm','https://www.geeksforgeeks.org/', 'https://www.geeksforgeeks.org/python-programming-language/?ref=shm', 'https://www.geeksforgeeks.org/fundamentals-of-algorithms/?ref=shm','https://www.geeksforgeeks.org/', 'https://www.geeksforgeeks.org/python-programming-language/?ref=shm', 'https://www.geeksforgeeks.org/fundamentals-of-algorithms/?ref=shm']
        
        for i in endpoints:
            tasks.append(asyncio.ensure_future(get_GFG_task(session, i)))
            
        web_responses = await asyncio.gather(*tasks)

        for web_response in web_responses:
            d[web_response] = web_response
            
    b = time.perf_counter()
    print(b - a, 'seconds')
    
asyncio.run(getGFGAsync())

For our async Python implementation we need to make a few changes to our code. First, we need to use aiohttp for our requests instead of the Requests library. Requests does not support async out of the box.

Next, we have to separate our request into an async function using the async keyword in the function definition. Then, we need to return a response using the await keyword. From there, we can convert the main function to async. Additionally, we’ll use the ensure_future method to call our helper function and append the response to our list. Finally, we can update our dictionary with the list of results.

Result: Running Python Asynchronously

Wow, after running our tasks concurrently we’ve reduced our processing time significantly. This example is fairly simple and doesn’t exactly require running Python asynchronously as the execution time is only seconds. However, if an application needs to request and process large payloads, async is a must! Not only for fast processing but reduced computing costs.

Need more help?

Contact Me