Multithreading Testing using Pythons’ Low Level _threading Module

In Python, we can use _threading to launch a thread easily using the _thread.start_new_thread procedure. For example,

1
2
3
4
5
6
7

import _thread
 
def thread_proc(threadId, value):
  print(threadId, value)
 
_thread.start_new_thread( thread_proc, ("Thread-1", "a Number") )
_thread.start_new_thread( thread_proc, ("Thread-2", "a Number") )

import _thread

def thread_proc(threadId, value):
  print(threadId, value)

_thread.start_new_thread( thread_proc, ("Thread-1", "a Number") )
_thread.start_new_thread( thread_proc, ("Thread-2", "a Number") )

Unfortunately, the above threads may not finish (and be aborted) before the main script is terminated. Because we are not synchronize the threads yet. We can however, do an easy trick:

1
2

while True:
  pass

while True:
  pass

This endless loop will allow all threads to forcibly joining but the script hangs until we Ctrl+C or kill it. We can use the threading module but that requires us to write a Thread class that inherits the threading.Thread.

We can uset the threading.Event() to join the threads. For example:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

import _thread
import threading
 
def thread_proc(evt, threadId, value):
  evt.set()
  print(threadId, value)
 
evt1 = threading.Event()
evt2 = threading.Event()
 
_thread.start_new_thread( thread_proc, (evt1, "Thread-1", "a Number") )
_thread.start_new_thread( thread_proc, (evt2, "Thread-2", "a Number") )
 
evt1.wait()
evt2.wait()

import _thread
import threading

def thread_proc(evt, threadId, value):
  evt.set()
  print(threadId, value)

evt1 = threading.Event()
evt2 = threading.Event()

_thread.start_new_thread( thread_proc, (evt1, "Thread-1", "a Number") )
_thread.start_new_thread( thread_proc, (evt2, "Thread-2", "a Number") )

evt1.wait()
evt2.wait()

Multithreading Requests to API Server using Python’s _threading Module

Let’s launch 100 threads that sends concurrent requests to a Steem API Node. And we need to store the threading.Event() in an array so that we can join all threads.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

import _thread
import threading
import json
import requests
from random import randrange
 
def worker(evt, threadName, block):
  data = {"jsonrpc":"2.0", "method":"condenser_api.get_block", "params":[block], "id":1}
  r = requests.post(url="https://api.steemit.com",json=data)
  rjson = r.json()
  result = rjson["result"]
  print(threadName, len(result["transactions"]))
  evt.set()
      
try:
  threads = []
  for i in range(100):
    evt = threading.Event()
    threads.append(evt)
    _thread.start_new_thread( worker, (evt, "Thead-" + str(i), randrange(1, 40000000)) )
  for i in threads:
    i.wait()
except:
  print("Error2")   

import _thread
import threading
import json
import requests
from random import randrange

def worker(evt, threadName, block):
  data = {"jsonrpc":"2.0", "method":"condenser_api.get_block", "params":[block], "id":1}
  r = requests.post(url="https://api.steemit.com",json=data)
  rjson = r.json()
  result = rjson["result"]
  print(threadName, len(result["transactions"]))
  evt.set()
      
try:
  threads = []
  for i in range(100):
    evt = threading.Event()
    threads.append(evt)
    _thread.start_new_thread( worker, (evt, "Thead-" + str(i), randrange(1, 40000000)) )
  for i in threads:
    i.wait()
except:
  print("Error2")   

As expected, it will show the following:

Thead-28 18
Thead-24 12
Thead-61 5
Thead-36 21
Thead-34 45
Thead-35 40
Thead-56 26
Thead-91 3
Thead-12 2
Thead-60 33
Thead-14 17
Thead-39 0
Thead-2 36
Thead-81 42
Thead-41 17
Thead-69 0
Thead-17 30
Thead-45 33
Thead-53 3
Thead-89 16
Thead-51 0
Thead-26 2
Thead-15 55
Thead-47 33
Thead-66 0
Thead-18 21
...
...

I have also tried other nodes, and the result seems to me that all nodes can handle multiple requests at the same time from the same origin.

–EOF (The Ultimate Computing & Technology Blog) —