Why does single-metric load balancing pick the wrong service under heavy traffic?
Traditional load-balancing approaches usually reduce algorithm choice to a single-variable list: distribute in turn, send to the fewest connections, pin by source address, or apply weights. This model may look adequate under normal traffic; but the moment a campaign, an API burst, streaming traffic, or a sudden user surge sets in, it stays weak as the sole basis for a decision.
For example, if two backends report the same response time, traffic splits arbitrarily when the system has no way to make a second comparison. A service with a low response time but a full queue can be selected; a service that appears to have few connections but is already generating errors can remain a candidate.
This indecision shows up most at the latency tail. While the average response time looks acceptable, the user experience can degrade at the high percentiles. And if the latency added by security, access control, or application-layer checks is not counted in, candidates that look fast but actually behave slowly can rise to the front.
The right model has to read live service signals at decision time and automatically eliminate unhealthy or in-maintenance candidates. When session affinity is active, it must not disturb the existing user binding; only on free requests should it select the most suitable service.
That is exactly the core problem Fastest+ Routing solves: binding the load-balancing decision not to a single label, but to real-time, two-stage service behavior.
Our approach
TR7 handles Fastest+ Routing with live-signal reading, two-stage scoring, health awareness, and interface-based configuration.
Every request is decided with live service data
The decision logic runs embedded in the data plane and reads live service statistics on each HTTP or TCP request. The selected service is written to the routing variable and traffic is forwarded according to that decision. The operator gets a dynamic per-request selection without writing any special code.
The second signal separates the remaining tied candidates
In the first stage, the services with the best value on the primary signal enter the candidate list. When several services share the same score, the second stage brings in the secondary signal. This is how practical decisions such as "at equal response time, pick the one with the emptier queue" are carried out within a single algorithm.
Health status and session binding are protected together
Services in maintenance mode or with an unsuitable health status are not admitted to the candidate list. When dynamic session affinity is active, the Fastest+ selection is skipped and the existing session binding is preserved. This approach does not disturb user sessions while optimizing for performance.
Decision signals are selected from the interface, no code required
The operator only picks the primary and the secondary decision signal. The system generates the required routing lines automatically during the configuration-generation phase. Complex decision logic becomes a simple policy choice for day-to-day operations.
Capabilities
Fastest+ Routing provides advanced routing capabilities that jointly evaluate live performance, health, and session binding across backends.
Two-stage minimum selection breaks equal scores in a controlled way
When a new best value is found on the primary decision signal, the candidate list is reset; services sharing that value are added to the list. In the second stage these candidates are narrowed again by the secondary signal. In the default configuration Fastest+ prefers, among the lowest-response-time candidates, the service with the lower current session load. This favours the less-loaded service among those that appear fast.
Traffic is decided with understandable service signals, not raw counters
The operator selects two decision signals for Fastest+: average response time, connection establishment time, queue wait time, current session load, current queue depth, connection errors, client-side error rate, server-side error rate, service-side aborts, or used connection capacity. These signals are mapped internally to the corresponding live service counters; the user never deals with raw metric names. This way traffic can be sent not just to the service that "looks fast," but to the backend that is healthier, emptier, or less error-prone at that moment.
Single-signal mode offers a simpler routing when needed
In service pools where a second comparison is not necessary, single-signal mode can be used. In this mode the decision is made only by the best value of the selected primary signal. No unnecessary decision depth is added for simpler service groups. The operator can manage both single-signal and two-signal modes from the same interface.
Maintenance and health status are filtered automatically during the decision
If a backend is in maintenance mode or its health status is unsuitable, it does not enter the candidate list. This behavior is embedded in the algorithm; it requires no extra manual rule, no extra access-control list, and no operational intervention. Planned maintenance, draining a problematic service, and temporary service removal affect the traffic decision automatically. The system selects only among suitable candidates.
When session affinity is active, Fastest+ does not disturb the existing binding
When the session affinity conditions are met, the Fastest+ selection is disabled. The request is routed to the service determined by the existing sticky-session logic. The generated routing condition is configured by the system transparently. This way performance optimization does not override the continuity of the user session.
The same decision model can be applied to HTTP and TCP services
Fastest+ can run both in the HTTP request phase and in the TCP content-inspection phase. For TCP services, the inspection window required for the decision is generated automatically. This way application-layer traffic and TCP-based services are handled within the same management model. The operator does not have to learn a different product logic per service type.
Clients without a sticky session go dynamically to the most suitable service
When dynamic affinity is used, Fastest+ runs only for clients that do not yet have a pinned session. While existing sessions are preserved, new or free requests are distributed by live signals. This model offers a balanced approach for both user continuity and instantaneous capacity use. Especially under heavy and fluctuating traffic, it contributes to a more efficient use of the service pool.
If no healthy candidate is found, traffic does not fall into a black hole
If no suitable backend is found during the decision, the special routing decision stays empty. In that case the default load-balancing behavior takes over. In situations where the system cannot produce a special routing decision, it falls back to the known default behavior instead of dropping traffic into the void. This fallback logic reduces operational risk.
Operational depth
Fastest+ Routing is not just an algorithm choice; it is designed together with high availability, visibility, reload, and integration behavior.
Action registration model
The decision logic is loaded when the service starts and defined as two separate actions: single-signal and two-signal. The single-signal action works with one decision input, the two-signal action with two decision inputs. The actions can be used in the HTTP and TCP request phases.
Low decision cost
Service statistics are read from an in-memory table; no extra socket, file read, or external query is required. The decision process performs a linear scan over the number of services. This structure keeps the routing decision close to the data plane even in pools with numerous backends.
Cluster failover behavior
In a two-node high-availability install, each node runs the same algorithm independently. Because live signals are evaluated node-locally, the new active node keeps making decisions with the statistics it observes itself after a failover. There is no dependency on an external shared score store.
Audit and visibility
The selected backend is held in the routing variable and can be added to the log format. This makes it possible to investigate retrospectively which request was routed to which service. Operations teams can see traffic decisions not just as an outcome, but as a traceable routing trail.
Reload behavior
During a soft reload the decision context is reloaded and the algorithm starts with a fresh memory state. Because no historical response-time observation is carried over, the first requests' decisions are based on current instantaneous data. The fact that configuration changes can be applied without restarting the pool reduces operational disruption.
WAAP and Layer-4 boundaries
If the WAAP layer blocks a request, Fastest+ is not invoked, so no unnecessary service selection is made. Fastest+ applies only in HTTP and TCP service pools. In Layer-4 services, the platform's Layer-4 algorithm options are used.
Where it is used
E-commerce traffic distribution during campaigns
In intense sales periods many backends receive traffic at once. Fastest+ evaluates signals such as response time and queue depth together, preventing fast-but-full-queue services from rising unnecessarily to the front. The result is a more balanced service use and a more controlled user experience.
Error-aware routing for financial API services
In financial API layers a fast response alone is not enough; error-generation behavior must also be part of the decision. Fastest+ can bring services with a low server-side error rate to the front and, on a tie, consider the current session load. This structure enables a more deliberate distribution for critical API traffic.
Load and speed selection at media edge nodes
In streaming and media traffic, the least-loaded node does not always give the best user experience. Fastest+ evaluates used connection capacity and response time together, weighing both the current connection use and the response behavior. This achieves a more precise traffic distribution between edge services.
Smart differentiation in multi-tenant SaaS service groups
In multi-tenant structures, each tenant's service group can behave differently. Fastest+ can favour the more stably running services using signals such as service-side aborts and connection establishment time. This approach makes tenant-based service quality more manageable.
Frequently asked questions
Which signals can be selected as the primary and secondary decision?
Can I use Fastest+ in single-signal mode when one signal is enough?
How is a service in maintenance mode treated by Fastest+?
Does Fastest+ take over when session affinity is active?
Does the same algorithm run on HTTP and TCP services?
How is decision quality affected after a soft reload?
Free your load-balancing decision from a single metric
Two-stage traffic distribution, configured with live signals and without writing code. Let's walk through it in a live setup on your own services.