HBASE-29850 Add support for dynamic per-request attributes in AsyncTable

[krconv]

Summary

Adds RequestAttributesFactory to AsyncTableBuilder for generating request attributes dynamically per-request.

Motivation

It is possible to do this for the HBase 2 Table client by overriding the RpcControllerFactory on the connection, but that doesn't work reliably with AsyncTable because retries happen on Netty threads, losing thread-local context. This change provides a hook that is guaranteed to be called on the initiating thread.

Changes

• Add RequestAttributesFactory interface with a single create(Map<String, byte[]>) method
• Add setRequestAttributesFactory() to AsyncTableBuilder
• Factory is invoked at the start of each operation (get, put, scan, batch, etc.)
• getRequestAttributes() returns static attributes; factory is only used for actual requests

Usage

AsyncTable<?> table = conn.getTableBuilder(tableName)
  .setRequestAttribute("static.key", value)
  .setRequestAttributesFactory(attrs -> {
    Map<String, byte[]> newAttrs = new HashMap<>(attrs);
    newAttrs.put("dynamic.key", getValueFromThreadLocal());
    return newAttrs;
  })
  .build();

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 28s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
+1 💚	hbaseanti	0m 0s		Patch does not have any anti-patterns.
			_ master Compile Tests _
+0 🆗	mvndep	0m 13s		Maven dependency ordering for branch
+1 💚	mvninstall	3m 27s		master passed
+1 💚	compile	4m 25s		master passed
+1 💚	checkstyle	1m 18s		master passed
+1 💚	spotbugs	2m 24s		master passed
+1 💚	spotless	0m 53s		branch has no errors when running spotless:check.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 12s		Maven dependency ordering for patch
+1 💚	mvninstall	3m 7s		the patch passed
+1 💚	compile	4m 23s		the patch passed
+1 💚	javac	4m 23s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	1m 18s		the patch passed
+1 💚	spotbugs	2m 37s		the patch passed
+1 💚	hadoopcheck	12m 25s		Patch does not cause any errors with Hadoop 3.3.6 3.4.1.
+1 💚	spotless	0m 48s		patch has no errors when running spotless:check.
			_ Other Tests _
+1 💚	asflicense	0m 19s		The patch does not generate ASF License warnings.
		47m 4s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7665/1/artifact/yetus-general-check/output/Dockerfile
GITHUB PR	#7665
Optional Tests	dupname asflicense javac spotbugs checkstyle codespell detsecrets compile hadoopcheck hbaseanti spotless
uname	Linux 65799be78773 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / ba61a0c
Default Java	Eclipse Adoptium-17.0.11+9
Max. process+thread count	86 (vs. ulimit of 30000)
modules	C: hbase-client hbase-server U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7665/1/console
versions	git=2.34.1 maven=3.9.8 spotbugs=4.7.3
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

[Apache-HBase]

🎊 +1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 34s		Docker mode activated.
-0 ⚠️	yetus	0m 3s		Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --author-ignore-list --blanks-eol-ignore-file --blanks-tabs-ignore-file --quick-hadoopcheck
			_ Prechecks _
			_ master Compile Tests _
+0 🆗	mvndep	0m 14s		Maven dependency ordering for branch
+1 💚	mvninstall	3m 14s		master passed
+1 💚	compile	1m 19s		master passed
+1 💚	javadoc	0m 47s		master passed
+1 💚	shadedjars	5m 51s		branch has no errors when building our shaded downstream artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 14s		Maven dependency ordering for patch
+1 💚	mvninstall	2m 56s		the patch passed
+1 💚	compile	1m 19s		the patch passed
+1 💚	javac	1m 19s		the patch passed
+1 💚	javadoc	0m 44s		the patch passed
+1 💚	shadedjars	5m 47s		patch has no errors when building our shaded downstream artifacts.
			_ Other Tests _
+1 💚	unit	1m 29s		hbase-client in the patch passed.
+1 💚	unit	234m 20s		hbase-server in the patch passed.
		263m 41s

Subsystem	Report/Notes
Docker	ClientAPI=1.43 ServerAPI=1.43 base: https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7665/1/artifact/yetus-jdk17-hadoop3-check/output/Dockerfile
GITHUB PR	#7665
Optional Tests	javac javadoc unit compile shadedjars
uname	Linux 7e5581651594 5.4.0-1103-aws #111~18.04.1-Ubuntu SMP Tue May 23 20:04:10 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/hbase-personality.sh
git revision	master / ba61a0c
Default Java	Eclipse Adoptium-17.0.11+9
Test Results	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7665/1/testReport/
Max. process+thread count	4881 (vs. ulimit of 30000)
modules	C: hbase-client hbase-server U: .
Console output	https://ci-hbase.apache.org/job/HBase-PreCommit-GitHub-PR/job/PR-7665/1/console
versions	git=2.34.1 maven=3.9.8
Powered by	Apache Yetus 0.15.0 https://yetus.apache.org

This message was automatically generated.

[Apache9]

So the requirements here is to use different request attribute for different requests?

I think the design of the request attribute API is for static attributes, if we want to support dynamic attribute, we'd better redesign the APIs, like what you propose here, use a factory.

Out of interest, what is your usage for this feature?

Thanks.

[krconv]

Yes—our requirement is to set a specific request attribute to a value calculated from thread-local context.

We use hbase.quota.user.override.key (which configures throttling to use a request attribute) to throttle based on a logical “upstream caller” rather than the connection user. For example, a high-volume nightly job (EmailJobs-nightlyPurgeJob) calls ObjectsWebService-web, which in this case is the component that actually holds the HBase AsyncTable client and issues deletes to a table objects-1. The service propagates the job identity via thread-local HTTP metadata and sets it as a request attribute so that the job is throttled independently, and can be slowed down without impacting user traffic and other lower-volume jobs. This matters because we share a single AsyncTable per table/JVM—without dynamic attributes, one noisy caller can cause throttling across all traffic handled by that service.

This is exactly what we do with the Table client using the RpcControllerFactory as a workaround, and it works well in our experience. With the AsyncTable though, the RpcControllerFactory is called from various threads during retries, so we don't have a way to propagate any thread-local context reliably.