Multithreading

Content Platform Tenant Management Help

Version
9.7.x
File Size
4269 KB
Audience
anonymous
Part Number
MK-95HCPH002-19

HCP lets multiple threads access a bucket concurrently. Using multiple threads can enhance performance, especially when accessing many small objects across multiple folders.

Here are some guidelines for the effective use of multithreading:

  • Concurrent threads, both reads and writes, should be directed against different folders. If that’s not possible, multiple threads working against a single folder is still better than a single thread.
  • To the extent possible, concurrent threads should work against different IP addresses. If that’s not possible, multiple threads working against a single IP address is still better than a single thread.
  • Only one client can write to a given object at one time. Similarly, a multithreaded client cannot have multiple threads writing to the same object at the same time. However, a multithreaded client can write to multiple objects at the same time.
  • Multiple clients can read the same object concurrently. Similarly, a multithreaded client can use multiple threads to read a single object. However, because the reads can occur out of order, you generally get better performance by using one thread per object.

The S3 compatible API shares a connection pool with the REST, and WebDAV APIs. HCP has a limit of 255 concurrent connections from this pool, with another 20 queued.

Tip: For better performance, consider limiting the number of concurrent read threads per node to 200 and concurrent write threads per node to 50 for small objects. For large objects, consider using fewer threads.