0 / 0

Web client connection

Last updated: Aug 01, 2025
Web client connection

Use a Web client connection to access data at an HTTP endpoint from a StreamSets flow. This connection can only be used in StreamSets flows.

Create a connection to the web client

To create a connection to use in StreamSets flows, you need the following details:

  • Base URL: Fixed base URL or request endpoint of the requested resource. For example, http://localhost:8000 or http://localhost:8000/basic/authenticate.
  • Request Headers: Additional common headers to include in the request.
  • Select Server proxy To access the web data source through a server proxy. Depending on its setup, a server proxy can provide load balancing, increased security, and privacy. The server proxy settings are independent of the authentication credentials and the personal or shared credentials selection. The server proxy settings cannot be stored in a vault.
    • Proxy Server: The proxy server endpoint.
    • The Proxy User and Proxy Password: The proxy username and password to access the endpoint.

Credentials

Authentication schema

Choose the authentication method for your web client connection and provide the needed details:

Basic
Username and password: The username and password needed to access the data source.
Bearer
Token: Provide the bearer token to access the data source.
Digest
Username and password: The username and password needed to access the data source.
None
You do not have an authentication method for the data source.
OAuth 1
Consumer key: Name of the OAuth consumer key.
Consumer secret: OAuth consumer secret.
Access token: OAuth 1.0 access token.
Access secret: OAuth 1.0 token secret.
OAuth 2
Grant type: Grant type required by the HTTP service.
  • Access token: URL to request the access token.
  • Client credentials: Client credentials that the HTTP service uses to identify the HTTP client.
  • Owner credentials: Resource owner credentials.

Keystore management

Choose the method that keystores for authenticating and encrypting to the HTTP server will be handled:

Automatic
The data source will automatically handle the keystores needed.
Manual
Manually set the keystore information by filling out the required fields:
  • Keystore location: Where to load the keystore from.
  • Keystore key algorithm: Algorithm to manage the keystore.
  • Truststore location: Path to the truststore file.
  • Truststore key algorithm: Algorithm to manage the truststore.
  • Default protocol versions: set this to only use modern default secure protocols, or set the transport protocol versions manually.
  • Default cipher suites: Set this to only use modern default cipher suites, or set the cipher suites manually.

Choose the method for creating a connection based on where you are in the platform

In a project
Click Assets > New asset > Connect to a data source. See Adding a connection to a project.
In a deployment space
Click Import assets > Data access > Connection. See Adding data assets to a deployment space.
In the Platform assets catalog
Click New connection. See Adding platform connections.

Next step: Add data assets from the connection

Parent topic: Supported connections