Four situations account for most evaluations
Use cases
Each one is set out the same way: the situation, what the gateway does, what you get, and what to measure in a pilot. The arithmetic from a real sizing is at the end.
Small-object workloads on a store you already own
The situation. A team runs an erasure-coded object store, and its workload is small objects: telemetry samples, log batches, imaging series, backup catalogues. Capacity is adequate. Object count is not. Listing has slowed, tail latency has risen, and a rebuild after a device failure takes longer than the runbook says it should.
What triggers this evaluation
- Listings that time out
- Their cost follows the number of keys examined, not the data returned.
- Tail latency before capacity
- The object index has outgrown memory, so lookups reach storage devices.
- Rebuilds that overrun
- Recovery and rebalancing proceed object by object.
What the gateway does
Arcois goes in front of the existing store. That store is registered as a backend, and clients are pointed at the gateway. Small objects are packed into sealed volumes, so the store receives few large objects in place of many small ones. Names, versions, listings and metadata are answered from the metadata plane. Buckets that already exist are brought under the gateway by an import operation. Nothing on the client side changes apart from the address.
What you get
Your object count keeps growing. Your backend's object count grows at a fraction of that rate, and every cost that follows count recedes with it: index size, per-object placement work, listing traversal, and the object-by-object passes that recovery and rebalancing run. The store keeps serving bytes and is not replaced. The remaining per-object record sits on a plane that grows by adding nodes.
What to measure in a pilot
The packing ratio your own size distribution reaches, the compaction backlog under your delete pattern, and the metadata plane's growth against your object-version count. Those three decide the economics, and all three are observable within a pilot.
Regional or sovereign S3 as a service
The situation. A provider sells storage in a named jurisdiction, under local law, with local support. It has racks, network and an object store. What it lacks is the platform between that store and the customer: isolation, per-tenant identity, quotas, metering it can invoice against, and self-service administration.
Why building it is the wrong project
- Each item is a distributed-systems problem
- Policy evaluation has to default to denial, so a missing rule cannot become an accidental grant.
- Quota accounting has to stay correct
- Across concurrent writes, restarts and node failures.
- Usage records have to survive failure
- A lost record is lost revenue.
What the gateway does
Each customer becomes a realm: an isolated namespace holding its own buckets, credentials, users and policies. The customer administers all of it, using the standard command-line tools or a console of its own. Evaluation is default-deny, and an explicit denial overrides an allowance granted by any other policy or by bucket ownership. Quotas bound a realm's stored bytes and object count. A daily record per realm feeds billing.
What you get
A sellable service rather than raw buckets, without the multi-year programme that assembling isolation, identity, quotas and metering separately implies. You keep your own hardware and your jurisdiction argument. Tenants are added on the store already installed, and further stores are registered as capacity requires. One identity model, one policy model and one metering record cover every tenant on every registered store. The service grows without a second operating model to learn.
What to measure in a pilot
Run a tenant onboarding end to end without operator involvement: realm, quota, the tenant creating its own users and keys, a write refused at the hard quota, and the daily usage record landing in whatever you invoice from. If that loop closes, the platform question is answered.
A provider exit, or consolidation with a way back
The situation. An organisation wants to leave a provider, or to consolidate several onto one. There is a lot of data, the price rise is real, and the conventional answer is a maintenance window that grows with it. The decision has been deferred at each renewal because the alternative implies downtime.
Where the leverage comes from
- A negotiation is bounded by the alternative
- When the alternative implies downtime, a retrieval bill and a retrained team, an incumbent can price accordingly.
- Price protection is the ability to leave
- Rather than contract wording.
What the gateway does
Both the old and the new provider are registered as backends, and clients address the gateway. A cutover operation copies the bucket's bytes to the new backend and updates the pointer, phase by phase, while clients keep reading and writing. The plan is held in the metadata plane, so a restart resumes it. A phase an operator must see first is a checkpoint, and the operation waits there for approval. The operation never deletes the old backend's bytes.
What you get
A provider change that is a scheduled, reversible operation rather than a project with an outage attached. The endpoint clients address does not change, so nothing in your applications is reconfigured for the move. The old bytes stay in place, so abandoning the move part-way is a decision rather than a second migration.
What to measure in a pilot
Cut one real bucket over, then cancel a second one half-way and confirm the old backend is exactly as it was. The second test is the one that turns an exit from a plan into a priced option.
You already have a good array
The situation. You bought well. The array handles small objects, it holds the object counts you need, and nobody is unhappy with it. Most of what you have read so far does not apply to you.
Good. None of what follows asks you to replace it.
What an array cannot do for you
- Tier you onto someone else's disk
- No vendor will move your cold data off its own hardware. It is not a technical limit.
- Keep a copy it does not control
- Replication goes to another of its own systems, running the same software.
- Make itself easy to leave
- Nor should you expect it to.
What the gateway does
It goes in front of the array you keep, and adds the things that were never about the array. Your array carries on doing exactly what it does well. The gateway holds the names, so what you build around the array stops being fixed in place.
Cold data belongs somewhere cheaper
Your array is the most expensive place you own. A lifecycle rule moves what nobody reads onto commodity capacity — a different make, a different vendor — and the application never notices. No vendor will tier you off its own hardware.
A copy your vendor cannot lose for you
Same-vendor replication survives a fire. It does not survive a bad firmware release, a mistake in the management plane, or ransomware that reaches it. A copy on a different make does.
Model upgrades stop being projects
Moving from a performance model to a capacity model is still a migration with an endpoint change. Through the gateway it is a background copy and a pointer update, with clients reading and writing the whole time.
One policy, one audit trail, one set of charts
Two arrays means two identity models, two audit trails and two access reviews. One retention rule expressed twice is one retention rule you cannot prove. The gateway collapses all of it into one.
Quotas that refuse
Most arrays report a quota. This one declines the write, and never declines a delete. If you bill for storage, that is the difference between an invoice and a loss.
The next petabyte can come from anyone
You do not have to move to benefit from being able to. Having the option is what changes the price of the array you keep.
What to measure in a pilot
Register the array you have, put one workload behind the gateway, and register a second target of commodity capacity. Then do the two things the array cannot: age one bucket down onto the cheap target on a schedule, and keep a mirrored copy of another on the second make. If both work without touching the application, the argument is settled.
What the arithmetic looked like on a real one
The first of those four situations, with real numbers on it. These are the figures from a customer sizing we ran: twenty billion objects, and a size distribution typical of any system storing documents, scans and log batches. The comparison is against a Ceph cluster built to hold them, because that is the sizing that was actually done.
How the twenty billion objects are sized
Three objects in four are under 10 KB. The average works out at about 86 KB, so twenty billion objects come to roughly 1.7 PB. The bytes are not the problem. The count is.
The figures assume every object is packed, and that a volume is sealed at a little over half full rather than waiting to fill. Fill the volumes and it goes further. Store the objects over 1 MB directly instead of packing them and it comes back to about 42 times, because those 472 million objects then count one each.
One line in that table is worth drawing out. A bucket index is not free metadata sitting beside the data — in several products it is made of objects itself. This sizing needed ten thousand buckets, each sharded sixty-four ways, to keep listing workable: 640,000 index objects before a single customer object was stored. Aggregation removes the reason for that fan-out, because the gateway lists from its own ordered rows. 32 million objects at the same five-million-per-bucket comfort is seven buckets, which at the same 64-way pre-shard is about 450 shards — and the backend no longer needs that pre-shard at all.
Those middle-column figures are Ceph's, then, not every product's. The shape of the problem is. Every S3 implementation spends an index entry and a placement decision per object, every one of them lists a bucket by walking keys rather than bytes, and every one of them has a point past which a single bucket stops being comfortable. What changes from product to product is where those walls sit. What does not change is that fewer objects means you meet them later, or not at all.
What the storage behind it has to be
Bring your own object count and we will work through the arithmetic
You already know your object count and size distribution. That is everything needed to work out what your backend would hold after aggregation, before anyone signs anything.