sharednothing

phase 000 / requirements

Start with constraints, not Kubernetes

The first architecture mistake is opening with tools. A systems-design conversation starts with what the product must do and what failure is acceptable. Assume a commerce-style platform with web and mobile clients, regional traffic, payments, inventory and order tracking.

My rule as CTO: technology is justified by a constraint. If I cannot point to the constraint, the tool is decoration.

phase 001 / estimation

Back-of-envelope sizing before architecture

At 50k requests/sec, suppose 20% hits Order, 10% Payment, and 70% is read-heavy Catalog traffic. If one Java pod safely sustains 350 requests/sec at the target p99, the edge needs roughly 143 pod-equivalents of application capacity before headroom. I would provision for 60–70% steady utilisation, not 95%, because autoscaling is not instantaneous.

The database estimate matters more than the pod count. Requests are cheap; state is not. I would separately estimate write QPS, row growth/day, retention, event throughput and the size of hot working sets for Redis.

phase 003 / decomposition

Service boundaries follow business capability

I would not create UserService, ProductService and OrderService merely because those are nouns in the schema. A service earns its boundary when it has a clear owner, data it controls, a distinct scaling/failure profile, and a release cadence that benefits from independence.

If two services share tables, deploy together and fail together, they are not independent services. They are a distributed monolith with network latency.

phase 004 / implementation

Inside a Java service: boring on purpose

For the service layer I would standardise on Java 21 + Spring Boot 3.x, with one small paved-road template. A service gets HTTP/gRPC adapters, domain/application code, persistence adapters, Kafka producers/consumers, OpenTelemetry, health endpoints and resilience defaults. Teams can vary the domain; they should not reinvent plumbing.

order-service/
  src/main/java/com/acme/order/
    api/          // REST controllers + DTOs
    application/  // use-cases, orchestration
    domain/       // aggregates, invariants, policies
    persistence/  // JPA/JOOQ repositories
    messaging/    // Kafka producer/consumer
    config/       // security, tracing, resilience

I prefer explicit ports/adapters over a giant “service” class because it keeps the domain testable without Spring. Use virtual threads only after measuring the workload; they simplify I/O concurrency but do not remove database or downstream capacity limits.

phase 005 / packaging

Docker is packaging, not architecture

Every service ships as an immutable image. Multi-stage builds keep the runtime small, images are scanned in CI, run as non-root, and configuration arrives at runtime. We never rebuild an image to move from staging to production.

FROM eclipse-temurin:21-jre
WORKDIR /app
COPY build/libs/order-service.jar app.jar
USER 10001
ENTRYPOINT ["java","-XX:MaxRAMPercentage=75","-jar","/app/app.jar"]

The image boundary gives us repeatability. It does not give us service boundaries, autoscaling, resilience or observability; those come from the application and platform design.

phase 006 / runtime

Kubernetes runtime model: what I actually care about

Kubernetes is the scheduler and reconciliation layer. A Deployment says how many replicas should exist; a Service gives stable discovery; HPA changes replicas using CPU plus workload metrics; a PodDisruptionBudget protects us during node maintenance; readiness decides whether a pod receives traffic; liveness is used sparingly because a bad liveness probe can create an outage by restarting healthy-but-slow processes.

I would use separate node pools for generic APIs versus special workloads, topology spread across availability zones, and resource requests based on measured p95 usage. Limits on memory are useful; CPU limits deserve care because throttling can create mysterious tail latency.

phase 007 / state

Data ownership is the real microservices boundary

Each service owns its database schema. No cross-service joins in production request paths, no direct writes into another service’s tables. PostgreSQL is the default transactional store because the operational cost is understood and its semantics are excellent. Redis is used for explicitly disposable hot data: catalog snapshots, rate-limit state, sessions where appropriate, and computed reads with clear TTLs.

For cross-service reads, choose deliberately between a synchronous API, a replicated read model, or analytics CDC. The right choice depends on freshness. Inventory reservation is synchronous and strongly consistent. Search indexing can lag. Finance reporting should leave the OLTP estate entirely and land in a warehouse through CDC/events.

phase 008 / async

Kafka is for facts that outlive the request

OrderPlaced, PaymentAuthorised and InventoryReserved are durable facts. They belong on Kafka because multiple consumers may care and the producer should not block on them. Commands are different: “reserve stock” has an intended owner. I keep commands explicit and events descriptive.

The hard part is not publishing; it is correctness around publishing. A database commit followed by a Kafka publish can split. The outbox pattern writes the domain change and an outbox row in one local transaction, then a relay publishes to Kafka. Consumers must be idempotent because delivery is at-least-once.

BEGIN;
UPDATE orders SET status = 'PLACED' WHERE id = :id;
INSERT INTO outbox(event_id, aggregate_id, type, payload)
VALUES (:eventId, :id, 'OrderPlaced', :json);
COMMIT;
-- relay publishes outbox row -> Kafka, then marks it delivered

phase 009 / failures

Design the failure path before the happy path

Every synchronous dependency gets a deadline, a bounded connection pool, bulkheading and metrics. Retries happen at one layer, with exponential backoff + jitter, and only for idempotent operations. Circuit breakers are a last line of defence, not a substitute for capacity planning.

For checkout, the critical path may be Gateway → Order → Inventory → Payment. Recommendations, email, loyalty and analytics do not belong there. If they fail, the purchase must still succeed. That single decision controls blast radius better than any service mesh.

Availability math is unforgiving: if five sequential dependencies are each 99.9% available and every one is required, the theoretical path is only about 99.5%. Independence alone does not create reliability; dependency depth destroys it.

phase 010 / operations

Observability is part of the architecture

Every inbound request gets a trace ID. OpenTelemetry propagates it across HTTP/gRPC and Kafka. Logs are structured, metrics follow RED for services and USE for infrastructure, and traces answer the cross-service question that logs cannot.

My alerting standard is SLO-first. “CPU 90%” is not automatically an incident; “checkout success below 99.95% and the error budget is burning 8×” is.

phase 011 / delivery

CI/CD exists to make change boring

A pull request runs unit tests, architecture tests, consumer-driven contract tests, dependency scanning, container scanning and an ephemeral integration test where it adds value. Main produces one signed image. Promotion changes configuration and deployment metadata, not the image.

Production rollout is progressive: small canary, compare SLOs, then expand. Database changes use expand/contract so old and new service versions overlap safely. Feature flags separate deployment from release; they also expire, because a permanent flag is just undocumented branching.

phase 012 / guardrails

Security should be a paved road, not a wiki page

Authentication happens at the edge, authorisation is rechecked in the domain service for sensitive operations. Workloads use short-lived identity rather than static credentials. Secrets live in a secret manager, not Git or container images. Kubernetes RBAC, namespaces and NetworkPolicies reduce the blast radius of a compromised pod.

The platform team should supply secure defaults: TLS, dependency policy, base images, logging redaction, SBOM generation, image signing and admission checks. Product teams should not need to become PKI experts to ship a feature safely.

phase 013 / restraint

What I would not build yet

I would not introduce a service mesh on day one, a workflow engine for every three-step flow, twenty databases because there are twenty services, or Kafka for request/response that is easier as HTTP. I would not split a 12-person team into 25 services because “Netflix does it”.

Complexity is a budget. Spend it where it buys deploy independence, failure isolation, scaling or compliance. Everything else should stay boring.

The likely evolution is modular monolith → a few high-value services → stronger platform automation → only then finer-grained decomposition where evidence says it pays.

phase 014 / operating model

The CTO view is ownership, economics and blast radius

I care about more than whether the diagram is technically correct. Each service needs one owning team, an SLO, an on-call path, a cost profile, documented dependencies and a deprecation story. If nobody owns a service, Kubernetes merely keeps an orphan highly available.

phase 015 / whiteboard

How I would explain this in an 8-minute design review

1. Clarify requirements: traffic, latency, availability, consistency, geography, compliance.
2. Estimate: peak RPS, write rate, data growth, cache working set, headroom.
3. Draw the edge: clients → gateway → small set of business services.
4. Assign ownership: every service owns its data and one team owns the service.
5. Keep sync shallow: only blocking decisions stay on the request path; side effects move to Kafka.
6. Choose storage: PostgreSQL for transactions, Redis for disposable hot reads, warehouse for analytics.
7. Runtime: Java/Spring Boot in Docker, Kubernetes for scheduling, scaling and progressive delivery.
8. Close with failure modes: timeouts, idempotency, retries, outbox, degraded mode, observability, SLOs and tradeoffs.

A senior answer is not the one with the most boxes. It is the one that explains why each box exists, what happens when it fails, and what you deliberately chose not to build.