Google Cloud Professional Data Engineer — Question 331

You need to give new website users a globally unique identifier (GUID) using a service that takes in data points and returns a GUID. This data is sourced from both internal and external systems via HTTP calls that you will make via microservices within your pipeline. There will be tens of thousands of messages per second and that can be multi-threaded. and you worry about the backpressure on the system. How should you design your pipeline to minimize that backpressure?

Answer options

• A. Call out to the service via HTTP.
• B. Create the pipeline statically in the class definition.
• C. Create a new object in the startBundle method of DoFn.
• D. Batch the job into ten-second increments.

Correct answer: D

Explanation

The correct answer is D because batching tasks into ten-second increments can help manage the flow of messages and reduce backpressure by allowing the system to process data in manageable chunks. Option A does not address backpressure effectively as continuous HTTP calls can overwhelm the system. Option B lacks flexibility and may not adapt to changing load conditions. Option C also does not provide a solution for managing high throughput and backpressure.