Build1 distinct publisher3 min readPublished
Co-hosting models on shared GPUs cut Salesforce's inference infrastructure bill eightfold, but the placement algorithm that made it cheap counted instances and ignored Availability Zones. SchedulingConfig reconciles the two.
The Engineer · Build desk

Compiled by The EngineerSomething wrong?How this is made
The AWS post's scope note matters: SchedulingConfig governs the placement plan for each individual scale operation [14]. Balance is therefore an outcome of each API call, not a property the endpoint holds. After enough scale-in and scale-out cycles, the per-operation plans can drift, and AWS's answer is a second setting in a second object: the endpoint's ScaleInPolicy with the CONSOLIDATION strategy, which runs a background sweeper that consolidates copies and releases idle instances while honoring AZ balance constraints [14][15]. Two-AZ placement is one parameter on the component and one policy on the endpoint.
AWS also leans on the phrase "best-effort," and it is worth sitting with. AWS describes availability-aware distribution as even placement across AZs on a best-effort basis, while stating separately that endpoint and inference component update operations persist multi-AZ placement, so HA survives model updates [16]. Persisting placement through updates is real engineering and the harder half of the problem. Best-effort spreading, though, is a behaviour to test on your own endpoint rather than a control to cite in an audit, particularly if the requirement is a hard internal rule like Salesforce's 2-AZ mandate for every production model [5].
Here is the arithmetic the post sets up but does not finish. The documented scenario is four instances, two per AZ, with four copies deployed under SPREAD, landing two in AZ-1 and two in AZ-2 [9]. Lose an AZ and two copies remain, which is half the serving capacity [1]. Two-AZ HA at unchanged throughput therefore costs double the copies you need in steady state, and the copies are the thing you were consolidating onto shared GPUs to save money on. Set MaxImbalance to 0 and the constraint tightens further: strict balance across two AZs admits only even copy counts [2].
The 8x number is Salesforce's, for Agentforce, and it comes specifically from co-hosting multiple models on shared GPUs [2]. For it to transfer you would need a fleet of models that each leave a GPU substantially idle, compatible enough accelerator and instance requirements to share, and traffic peaks that do not arrive together. One large model per endpoint gets none of it, and neither does a portfolio whose components all spike at the same hour. What transfers unconditionally is the failure mode: without SchedulingConfig, the default algorithm optimizes each deployment operation on its own and distributes copies evenly across instances without considering AZs, so copies of one model can pile into one zone even on a multi-AZ endpoint [3].
The post also tells you never to set CopyCount to 1 for an HA-critical model, because a single copy lives in a single AZ [13]. In plain terms, a single copy means your high-availability plan runs in one Availability Zone.
Ranked by verification strength, evidence, and original report placement.
Salesforce set out to make Agentforce, its AI foundation for agents, highly available across multiple Availability Zones, and faced a gap: SageMaker AI Inference Components could cut GPU costs but their default placement did not guarantee the multi-AZ resilience Salesforce's compliance bar required.
By default the SageMaker placement algorithm optimizes each Inference Component deployment operation independently, distributing new copies evenly across instances without considering AZ balance, so even on a multi-AZ endpoint copies of a specific model can end up unevenly distributed across AZs.
The post lists the resulting single points of failure: a single instance crash takes down all copies of a model, and an AZ outage makes the entire model unavailable.
Salesforce mandates 2-AZ support for every production model, and default Inference Component placement, optimized for cost alone, did not yet meet that internal 2-AZ compliance bar.
AWS introduced the SchedulingConfig parameter in the CreateInferenceComponent API, giving customers fine-grained control over Inference Component copy placement across instances and Availability Zones.
The AvailabilityZoneBalance sub-parameter controls cross-AZ distribution, balancing copies evenly across Availability Zones with configurable imbalance tolerance.
Distinct publishers with included, body-backed reporting in this cluster.
1 article · August 28, 2026
Follow any of these and your For You feed starts watching them — no settings page required.
invest
Salesforce's double digits, minus Informatica: agentic AI is real and still 2% of revenue1 distinct publisher
product
Salesforce's longer-dated backlog grows at half the rate of its cRPO headline1 distinct publisher
build
Bedrock turns GPT-5.6 throughput into a routing choice, with residency as the price1 distinct publisher
product
Salesforce's answer to the SaaSpocalypse: 11% growth, and a $2.6B mark on Anthropic1 distinct publisher
Evidence-backed comparisons of source perspectives and observed adoption signals. Read the methodology
Which Builder, Operator, and Investor concerns the observed source mix emphasized—not a truth score.
Evidence, demonstrated adoption, hype gap, incentives, and confidence are assessed independently, each on its own current evidence. How these are measured.
Authoritative on the API, untested on the outcome
The service owner is describing its own parameter, so the mechanics — SchedulingConfig, MaxImbalance semantics, SPREAD versus BINPACK, the CONSOLIDATION sweeper — are about as solid as evidence gets short of reading the code. Everything downstream of the mechanics is assertion: the 8x saving has no baseline, the 2-AZ compliance claim rests on Salesforce's own bar, and no measurement of a zone failure appears anywhere.
One flagship tenant, no second data point
Salesforce running Agentforce under a 2-AZ mandate is a genuine production reference rather than a preview demo, and the controls exist in a shipped API. But this reporting names exactly one adopter and one endpoint shape: no other customers, no fleet or region figures, no indication of how widely the placement controls are being used outside the account AWS chose to profile.
Restrained prose, one unguarded number
For vendor writing this is sober — it tells you not to set CopyCount to 1, admits the zone spread is best-effort, and sends you to a separate policy for long-run rebalancing. The overstatement is by omission. An 8x saving is repeated as the frame while the bill for the fix is never drawn: SPREAD deliberately uses more instances than packing, and a two-zone layout that survives a zone loss at full throughput needs roughly double the copies it steady-state serves with.
The API's owner and its showcase customer share the byline
AWS publishes this on its own machine learning blog, about its own scheduling parameter, using a name-brand customer whose story doubles as a reason to keep GPU workloads on SageMaker. Salesforce gets to look rigorous about resilience. No cloud rival, no independent operator, and no skeptical engineer appears anywhere in the story to price the tradeoff differently.
Narrow claims, solid; broad ones, take on trust
Confidence splits cleanly by claim type. Anything about parameter names and placement behaviour can be checked against the API within an hour and comes from the only party who would know. Anything about cost, compliance sufficiency, or real-world resilience rests on one self-interested telling with no numbers to interrogate, and the source text itself breaks off partway through its capacity guidance.