Load Balancing Across Back End Servers | Apigee | google cloud (2023)

you see they seecall himYApigee-HybridDocumentation.
vistashould apigeeDocumentation.

Apigee improves the availability of your APIs by providing built-in support for load balancing and failover across multiple backend server instances.

TargetServer decouples specific endpoint URLs from TargetEndpoint configurations. Instead of defining a specific URL in the configuration, you can configure one or more named TargetServers. Each TargetServer is then referred to by name in a TargetEndpoint-HTTPConnection.


Watch the following videos to learn more about API routing and load balancing with target servers.

Video Description
Load balancing with destination servers Load balancing API on target servers.
Environment-based API routing with destination servers Forward an API to a different destination server depending on the environment.

About TargetServer configuration

A TargetServer configuration consists of a name, host, protocol, and port with an additional element indicating whether the TargetServer is enabled or disabled.

The following code is an example of a TargetServer configuration:

{ "name": "target1", "host": "1.mybackendservice.com", "protocol": "http", "port": "80", "isEnabled": "true"}

For more information on the TargetServers API, seeOrganizations.Environments.Destination servers.

See TargetServer schema and other entitiesGitHub.

Create destination server

Create target servers using the Apigee UI or API, as described in the following sections.

Apigee-UI in the Cloud-Konsole

To create target servers using the Apigee UI in the cloud console:

  1. accessApigee-UI in the Cloud-Konsole.
  2. ChooseManagement > Environments > Destination servers.
  3. Select the environment in which you want to define a new TargetServer.
  4. Clickdestination serverat the top of the area.
  5. Click+ Create destination server.
  6. Enter a name, host, protocol, and port in the fields provided. The options forProtocolSonHTTPoGRPC.

    You can optionally selectHabilitar SSL. VerOverview of SSL certificates.

  7. ClickCreate

Classic Apigee UI

To create target servers with the classic Apigee UI:

(Video) How to Configure Backend Services for Global External HTTP(S) Load Balancer

  1. accessApigee User Interface.
  2. ChooseAdmin > Environments > Destination servers.
  3. Of theSurroundingsFrom the dropdown list, select the environment for which you want to define a new TargetServer.

    The Apigee UI displays a list of the current target servers in the selected environment:

    Load Balancing Across Back End Servers | Apigee | google cloud (1)

  4. Click+destination serverto add a new TargetServer to the selected environment.

    Headd destination serverThe dialog shows:

    Load Balancing Across Back End Servers | Apigee | google cloud (2)

  5. ClickPermittedto activate the new TargetServer. Definition just after creation.
  6. Enter a name, host, protocol, and port in the fields provided. The options forProtocolSonHTTPoGRPC.

    Optionally, you can select theSSLcheck box. For more information on these fields, seedestination server(4MV4D-Video).

  7. Clickadd.

    Apigee creates the new TargetServer definition.

  8. After creating a new TargetServer, you can edit its proxy API and the<HTTP destination connection>element to reference the new definition.

Apigee APIs

The following sections provide examples of using the Apigee API to create and enumerate TargetServers.

  • Create a TargetServer in an environment using the API
  • List target servers in an environment using the API

For more information, seeTargetServers-API.

Create a TargetServer in an environment using the API

To create a TargetServer calledalma1what CONECT1.miserviciobackend.comin the port80use the following API call:

curl "https://apigee.googleapis.com/v1/organizations/$ORG/environments/$ENV/targetservers" \ -X POST \ -H "Content-Type:application/json" \ -H "Autorización: Portador $TOKEN" \ -d ' { "name": "target1", "host": "1.mybackendservice .com", "Protocolo": "http", "Puerto": "80", "isEnabled": "verdadero", }'

Wo$FICHAis configured for your OAuth 2.0 access token as described inObtain an OAuth 2.0 access token. For information about theshirred ruffleFor the options used in this example, seewear curl. For a description of the environment variables used, seeSetting environment variables for Apigee API requests.

Here is an example of the response:

{ "host": "1.mybackendservice.com", "protocol": "http", "isEnabled": true, "name": "target1", "port": 80}

Create a second TargetServer with the following API call. By defining two TargetServers, you provide two URLs that a TargetEndpoint can use for load balancing:

curl "https://apigee.googleapis.com/v1/organizations/$ORG/environments/$ENV/targetservers" \ -X POST \ -H "Content-Type:application/json" \ -H "Autorización: Portador $TOKEN" \ -d ' { "name": "target2", "host": "2.mybackendservice .com", "Protocolo": "http", "Puerto": "80", "isEnabled": "verdadero", }'

Here is an example of the response:

(Video) What is Cloud Load Balancing?

{ "host": "2.mybackendservice.com", "protocol": "http", "isEnabled": true, "name": "target2", "port": 80}

List target servers in an environment using the API

To enumerate the TargetServers in an environment, use the following API call:

curl https://apigee.googleapis.com/v1/organizations/$ORG/environments/$ENV/targetservers \ -H "Authorization: Carrier $TOKEN"

Here is an example of the response:

[ "Objective2", "Objective1" ]

Two TargetServers are now available for use by API proxies provided inproofSurroundings. To balance traffic across these destination servers, configure the HTTP connection on the destination endpoint of an API proxy to use the destination servers.

Configure a TargetEndpoint to load balance between named TargetServers

Now that you have two TargetServers available, you can change the TargetEndpoint HTTP connection configuration to refer to these two TargetServers by name:

<TargetEndpoint name="predeterminado"> <HTTPTargetConnection> <LoadBalancer> <Servername="target1" /> <Servername="target2" /> </LoadBalancer> <Path>/test</Path> </HTTPTargetConnection>< /Zielendpunkt >

The configuration above is the most basic load balancing configuration. The load balancer supports three load balancing algorithms: round robin, weighted, and least connected. Round Robin is the default algorithm. Since no algorithm is specified in the above configuration, outgoing API proxy requests to back-end servers are executed one by one.alma1Ygol2.

He<ruta>The element forms the base path of the TargetEndpoint URI for all TargetServers. only used if<Load Balancer>is used. Otherwise, it will be ignored. In the example above, one request is enough.alma1it will behttp://ziel1/testand so for other TargetServers.

For more information, seedestination endpoint.

Configure load balancer options

You can optimize availability by configuring load balancing and failover options at the load balancer and target server level, as described in the following sections.


Specifies the algorithm used<Load Balancer>. The available algorithms areconcentricity,weighted, Ysmaller connections, each documented below.

all against all

The default algorithm, round robin, forwards a request to each destination server in the order that the servers appear in the destination endpoint's HTTP connection. For example:

<TargetEndpoint name="default"> <HTTPTargetConnection> <LoadBalancer> <Algorithm>RoundRobin</Algorithm> <Servername="Target1" /> <Servername="Target2" /> </LoadBalancer> <Path>/test</Path > </HTTPTargetConnection></TargetEndpoint>


The weighted load balancing algorithm allows you to configure proportional traffic loads for your destination servers. Weighted LoadBalancer distributes requests to its target servers in direct proportion to the weight of each target server. Therefore, the weighted algorithm requires you to set aWeightattribute for each TargetServer. For example:

<TargetEndpoint name="default"> <HTTPTargetConnection> <LoadBalancer> <Algorithm>Weighted</Algorithm> <Server name="target1"> <Weight>1</Weight> </Server> <Server name="target2"> <Weight>2</Weight> </Server> </LoadBalancer> <Path>/test</Path> </HTTPTargetConnection></TargetEndpoint>

In this example, two requests are forwardedgol2for each request sentalma1.

minimum connection

LoadBalancers configured to use the least connection algorithm forward outgoing requests to the TargetServer with the fewest open HTTP connections. For example:

<TargetEndpoint name="default"> <HTTPTargetConnection> <LoadBalancer> <Algorithmus>LeastConnections</Algorithm> <Servername="target1" /> <Servername="target2" /> </LoadBalancer> </HTTPTargetConnection> <Path >/ prueba</Path></TargetEndpoint>

maximum failures

The maximum number of failed API proxy requests to the TargetServer that result in the request being redirected to another TargetServer.

(Video) Cloud Load Balancing Deep Dive and Best Practices (Cloud Next '19)

A response error means that Apigee is not receiving a response from TargetServer. When this happens, the error counter increases by one.

However, when Apigee receives a response from a target, even if the response is an HTTP error (for example,500), which counts as a response from TargetServer and resets the error counter. To make sure that bad HTTP responses (such as500) also increases the error counter to get a failed server out of the load balancing rotation as soon as possible, you can use the<Bad response from server>article with<response code>add child elements to your load balancer configuration. Apigee also counts responses with these codes as errors.

In the following examplealma1will be removed from rotation after five failed requests, including some5XXAnswers from TargetServer.

<TargetEndpoint name="default"> <HTTPTargetConnection> <LoadBalancer> <Algorithm>RoundRobin</Algorithm> <Server name="target1" /> <Server name="target2" /> <MaxFailures>5</MaxFailures> <ServerUnhealthyResponse > <ResponseCode>500</ResponseCode> <ResponseCode>502</ResponseCode> <ResponseCode>503</ResponseCode> </ServerUnhealthyResponse> </LoadBalancer> <Path>/test</Path> </HTTPTargetConnection></TargetEndpoint>

The default value of MaxFailures is 0.This means that Apigee always attempts to connect to the target for each request and never removes the TargetServer from the rotation.

It is best to use MaxFailures > 0 with a HealthMonitor. If you set MaxFailures > 0, the TargetServer will be removed from rotation if the target fails the number of times you specified. When ahealth monitoris present, Apigee will automatically rotate the TargetServer again after the target is up and running again, as configured by this HealthMonitor. Seehealth surveillancefor more information

Alternatively, if you set MaxFailures > 0 and do not configure a HealthMonitor, Apigee will not automatically include the TargetServer back in rotation after Apigee detects an error. In this case, you must redeploy the API proxy before Apigee rotates TargetServer again. SeeProvide an API proxy.


When retry is enabled, a request is retried whenever a response error occurs (I/O error or HTTP timeout) or the received response matches a value specified by<Bad response from server>. Vermaximum failuresabove for more information on configuration<Bad response from server>.

Default<Retry Enabled>is set toTRUE. To placeINCORRECTturn off repeat. For example:


Ice Reserve

One (and only one) TargetServer can be configured asto fall againServer. The alternate destination server does not participate in load balancing routines until the load balancer has identified all other destination servers as unavailable. If the load balancer determines that all destination servers are unavailable, all traffic is forwarded to the standby server. For example:

<TargetEndpoint name="default"> <HTTPTargetConnection> <LoadBalancer> <Algorithm>RoundRobin</Algorithm> <Servername="target1" /> <Servername="target2" /> <Servername="target3"> <IsFallback >true< /IsFallback> </Servidor> </LoadBalancer> <Ruta>/prueba</Ruta> </HTTPTargetConnection></TargetEndpoint>

The above configuration results in round-robin load balancing betweenalma1Ygol2until bothalma1Ygol2Unavailable. Whetheralma1Ygol2are not available, all traffic is forwardedZiel3.


Path defines a URI fragment that is added to all requests sent from the TargetServer to the backend server.

This element accepts a literal string path or amessage template. A message template allows you to replace variable strings at runtime. For example, in the following destination endpoint definition, the value of{my way}is used for the path:

<HTTPargetConnection> <SSLInfo> <Enabled>true</Enabled> </SSLInfo> <LoadBalancer> <Servername="testserver"/> </LoadBalancer> <Path>{mypath}</Path></HTTPargetConnection>

Configuring a destination server for TLS/SSL

If you use a TargetServer to define the backend service, and the backend service requires the connection to use the HTTPS protocol, you must enable TLS/SSL in the TargetServer definition. This is necessary because thehostyou cannot specify the connection protocol. The following is the TargetServer definition for one-way TLS/SSL, where Apigee makes HTTPS requests to the backend service:

{ "name": "target1", "host": "mocktarget.apigee.net", "protocol": "http", "port": "443", "isEnabled": "true","sSLInfo": { "enabled": "true" }}

If the back-end service requires two-way or mutual TLS/SSL, configure TargetServer with the same TLS/SSL configuration settings as TargetEndpoints:

(Video) What is a Load Balancer?

{ "nombre": "TargetServer 1", "host": "www.example.com", "protocolo": "http", "puerto": "443", "isEnabled": "true", "sSLInfo": { "enabled": "true", "clientAuthEnabled": "true", "keyStore": "keystore-name", "keyAlias": "keystore-alias", "trustStore": "truststore-name", "ignoreValidationErrors" : "falsch", "Chiffren": [] }}

To create a TargetServer with SSL using an API call:

curl "https://apigee.googleapis.com/v1/organizations/$ORG/environments/$ENV/targetservers" \ -X POST \ -H "Content-Type:application/json" \ -H "Authorization: $TOKEN bearer" \ -d ' { "name": "TargetServer 1", "host": "www. example.com", "protocol": "http", "port": 443, "isEnabled": true, "sSLInfo": { "enabled": true, "clientAuthEnabled": true, "keyStore": "storename ", "keyAlias":"keystore-alias", "ciphers":[], "protocols":[], "clientAuthEnabled":true, "ignoreValidationErrors":false, "trustStore":"truststore-name" } }'

health surveillance

With health monitoring, you can improve your load balancing configurations by actively querying the back-end service URLs defined in the TargetServer configurations. With health monitoring enabled, destination servers that cannot be reached or return an error response are marked as unhealthy. A failed TargetServer is automatically put back into rotation when HealthMonitor detects that the TargetServer is up. No proxy redistributions required.

HealthMonitor acts as a simple client that calls a back-end service over TCP or HTTP:

  • A TCP client only guarantees that a socket can be opened.
  • Configures the HTTP client to send a valid HTTP request to the backend service. You can define HTTPRECEIVE,TO PLACE,CORREO, oEXTINGUISHoperations. The HTTP monitor call response must match the settings configured in the<Success response>Block.

successes and failures

When you enable health monitoring, Apigee starts sendinghealth checksto your destination server. TOhealth checkis a request sent to the TargetServer that determines if the TargetServer is healthy or not.

A status check can have one of two possible results:

  • Success:TargetServer is considered error free if a successful health check is performed. This is usually the result of one or more of the following:
    • TargetServer accepts a new connection on the specified port, responds to a request on that port, and then closes the port within the specified time frame. The response from the TargetServer containsconnection: close
    • TargetServer responds to a health check request with a200 (good)or any other HTTP status code you deem acceptable.
    • The TargetServer responds to an integrity check request with a message body that matches the expected message body.

    If Apigee determines that a server is healthy, Apigee will continue to send requests to it or continue.

  • Fail:Depending on the type of check, TargetServer can fail a health check in different ways. An error may be logged if TargetServer:
    • Denies Apigee's connection to the health check port.
    • You do not respond to a health check request within a certain period of time.
    • Returns an unexpected HTTP status code.
    • Responds with a message body that does not match the expected message body.

    When a TargetServer fails a health check, Apigee increments the failure count for that server. If the number of errors from this server reaches or exceeds a predefined threshold (<MaxFehler>), Apigee stops sending requests to this server.

Activate a HealthMonitor

To create a HealthMonitor, add the<Health Monitor>element in the HTTPConnection configuration of the TargetEndpoint for a proxy. You cannot do this in the user interface. Instead, it creates a proxy configuration and uploads it to Apigee as a ZIP file. A proxy configuration is a structured description of all aspects of a proxy API. Proxy configurations consist of XML files in a predefined directory structure. For more information, seeAPI proxy configuration reference.

Un HealthMonitor simple define unIntervaloEnSeccombined with a TCPMonitor or an HTTPMonitor. He<MaxFehler>The element specifies the maximum number of failed API proxy requests to the TargetServer that result in the request being redirected to another TargetServer. Default<MaxFehler>is 0, which means that Apigee does not take any corrective action. When setting up a health monitor, be sure to set<MaxFehler>soy<HTTP destination connection>day of<destination endpoint>label to a non-zero value.


The following configuration defines a HealthMonitor that polls each TargetServer by opening a connection on port 80 every five seconds. (The port is optional. If not specified, the TCPMonitor port is the TargetServer port.)

  • If the connection fails or takes more than 10 seconds to connect, the error counter for that destination server is incremented by 1.
  • If the connection is successful, the TargetServer error counter is reset to 0.

You can add a HealthMonitor as a child element of the HTTPTargetConnetion element of the TargetEndpoint, as shown below:

<TargetEndpoint name="default"> <HTTPTargetConnection> <LoadBalancer> <Algorithm>RoundRobin</Algorithm> <Server name="target1" /> <Server name="target2" /> <MaxFailures>5</MaxFailures> </ LoadBalancer> <Path>/test</Path> <HealthMonitor> <IsEnabled>true</IsEnabled> <IntervalInSec>5</IntervalInSec> <TCPMonitor> <ConnectTimeoutInSec>10</ConnectTimeoutInSec> <Port>80</Port> < /TCPMonitor> </HealthMonitor> </HTTPTargetConnection>. . .

HealthMonitor Configuration Items with TCPMonitor

The following table describes the TCPMonitor configuration items:

Name Description Standard Necessary?
activated A boolean value that turns HealthMonitor on or off. INCORRECT NO
IntervaloEnSec The time interval in seconds between each TCP probe request. 0 Y
ConnectTimeoutInSec The time in which the connection to the TCP port must be established to be counted as a success. If a connection is not established in the specified interval, this is counted as an error, which increases the load balancer error count for TargetServer. 0 Y
Puerto Optional. The port on which the TCP connection is established. If not specified, the TCPMonitor port is the TargetServer port. 0 NO

HTTP monitoring

A sample HealthMonitor using an HTTPMonitor sends aRECEIVERequest to the backend service once every five seconds. The following example adds a basic HTTP authorization header to the request message. The response configuration defines the configuration that is compared to the actual response from the backend service. In the following example, the expected response is an HTTP response code200and a custom HTTP headerI'm finewhose value isIhrOK. If the response does not match, the load balancer configuration treats the request as an error.

HTTPMonitor supports back-end services configured to use one-way HTTP and HTTPS protocols. However, it does not support two-way HTTPS, also known as two-way TLS/SSL.

(Video) Cloud OnAir: Networking 104 - Everything You Need to Know About Load Balancers on GCP

Note that all request and response settings in an HTTP monitor are specific to the backend service that needs to be invoked.

<HealthMonitor> <IsEnabled>true</IsEnabled> <IntervalInSec>5</IntervalInSec> <HTTPMonitor> <Request> <ConnectTimeoutInSec>10</ConnectTimeoutInSec> <SocketReadTimeoutInSec>30</SocketReadTimeoutInSec> <Port>80</Port> < Verb>GET</Verb> <Path>/healthcheck</Path> <Header name="Autorización">Básico 12e98yfw87etf</Header> <IncludeHealthCheckIdHeader>true</IncludeHealthCheckIdHeader> </Request> <SuccessResponse> <ResponseCode>200< /ResponseCode> <Header name="ImOK">IhrOK</Header> </SuccessResponse> </HTTPMonitor> </HealthMonitor>

HealthMonitor with HTTPMonitor configuration items

The following table describes the HTTPMonitor configuration items:

Name Description Standard Necessary?
activated A boolean value that turns HealthMonitor on or off. INCORRECT NO
IntervaloEnSec The time interval in seconds between each query request. 0 Y

Configuration options for the outgoing request message sent by HealthMonitor to the destination servers in the rotation.

The route does not support variables.

N / A Y
ConnectTimeoutInSec The time in seconds in which the handshake of the TCP connection to the HTTP service must complete to be considered successful. If the connection is not established within the specified interval, this is counted as an error, which increments the LoadBalancer error counter for TargetServer. 0 NO
SocketReadTimeoutInSec The time in seconds that data must be read from the HTTP service for it to be considered a success. If the specified interval is not read, this is counted as an error, which increments the LoadBalancer error counter for the TargetServer. 0 NO
Puerto The port on which the HTTP connection to the backend service is made. N / A NO
Verb The HTTP verb used for each HTTP query request to the backend service. N / A NO
Far The path attached to the URL defined in the TargetServer. Use the route element to configure a "probe endpoint" for your HTTP service. N / A NO


Allows you to track health check requests on upstream systems. HeIncluirHealthCheckIdHeaderIncluirHealthCheckIdHeadertakes a boolean value and is on by defaultINCORRECT. if you set itTRUE, then there is oneHeadercalledX-Apigee-Healthcheck-IDwhich is inserted into the Healthcheck application. The header value is dynamically assigned and takes the formORG/ENV/SERVER_UUID/N, isORG.is the name of the organization,ENVis the name of the environment,SERVIDOR_UUIDis a unique ID that identifies the MP, andnorteis the number of milliseconds that have elapsed since January 1, 1970.

Example of a resulting request header:

X-Apigee-Healthcheck-ID: orgname/envname/E8C4D2EE-3A69-428A-8616-030ABDE864E0/1586802968123
useful load The HTTP body generated for each query HTTP request. Note that this item is not required forRECEIVEPetitions. N / A NO
success answer Options matching for the incoming HTTP response message generated by the backend service being queried. Non-matching responses increase the error count by 1. N / A NO
response code The expected HTTP response code from the queried TargetServer. Any code other than the one specified will fail and the counter will be incremented for the queried backend service. You can define multiple ResponseCode elements. N / A NO
holders A list of one or more HTTP headers and values ​​expected by the backend service being queried. Any HTTP headers or values ​​in the response that differ from those specified will generate an error and the queried TargetServer's counter will be incremented by 1. You can define multiple header elements. N / A NO


1. Configuring an HTTP Load Balancer with Autoscaling || #qwiklabs || #coursera [With Explanation 🗣️]
(Quick Lab)
2. [#NEW ] - HTTP Load Balancer with Cloud Armor || #qwiklabs || #GSP215 || [With Explanation🗣️]
(Quick Lab)
3. What’s new for load balancing and service networking
(Google Cloud Tech)
4. How to troubleshoot unhealthy backends in Google Cloud Load Balancers
(Google Cloud Tech)
5. Google Cloud Platform (GCP) - HTTP Load Balancer
(Cloud Monkey)
6. Life of a photon through global load balancing
(Google Cloud Tech)


Top Articles
Latest Posts
Article information

Author: Annamae Dooley

Last Updated: 07/03/2023

Views: 6170

Rating: 4.4 / 5 (45 voted)

Reviews: 92% of readers found this page helpful

Author information

Name: Annamae Dooley

Birthday: 2001-07-26

Address: 9687 Tambra Meadow, Bradleyhaven, TN 53219

Phone: +9316045904039

Job: Future Coordinator

Hobby: Archery, Couponing, Poi, Kite flying, Knitting, Rappelling, Baseball

Introduction: My name is Annamae Dooley, I am a witty, quaint, lovely, clever, rich, sparkling, powerful person who loves writing and wants to share my knowledge and understanding with you.