MySQL 26.7.0 was released on July 28, 2026. Among its replication changes, MySQL introduced the Change Stream Applier, or CSA, a new implementation of the replica SQL applier for multithreaded replication. CSA is currently opt in, but some of its requirements are interesting: it does not support file position replication channels, requires GTID_MODE=ON, and requires GTID_ONLY=1.¹

What interested me was not GTID itself, but what the requirement implies: MySQL’s newer replication machinery is making transaction identity a prerequisite while leaving file position replication behind.

That made me ask what the equivalent recovery state actually is in Kafka.

File Position Versus Transaction Identity

Traditional MySQL replication can describe progress using a binary log file and byte position:

mysql-bin.004217:18372944

That answers:

What location should I continue from?

A Global Transaction Identifier answers a different question.

MySQL assigns a GTID to a transaction when that transaction is committed on its originating server. When the transaction is replicated, it retains the same GTID on the replica.²

Conceptually:

server-uuid:transaction-number

The transaction can appear in different binary log files and at different positions as it moves through the topology, while its identity remains unchanged.

With GTID-based replication, MySQL can reason about which transactions have already been executed and which are still missing without depending on binary log filenames and positions.³

The recovery question changes from:

Where was I?

to:

Which transactions have I already applied?

That distinction immediately reminded me of Kafka.

Kafka Uses Positions Differently

Kafka assigns each record a monotonically increasing offset within a topic partition:

orders
partition 7
offset 928471

The offset identifies a position in that partition’s log. It does not globally identify the business event.

Within a Kafka cluster, however, the offset is already independent of physical broker placement. A partition is replicated across brokers, and if leadership moves to another replica, consumers continue operating in terms of the same topic, partition, and offset space.⁴

So Kafka separates:

physical broker placement

from:

topic + partition + offset

Consumer progress adds another dimension.

Kafka does not maintain one consumer position for a topic. A committed position belongs to a consumer group and topic partition:

consumer-group
    +
topic
    +
partition
    →
committed offset

Kafka stores those committed group offsets in the internal compacted topic __consumer_offsets.⁵

For example:

group: payment-processor
topic: payments
partition: 3
committed offset: 10001

The committed offset normally represents the next record to consume. If the consumer successfully processed record 10000, it would typically commit 10001.⁶

Another consumer group reading the same partition may have a completely different committed position.

This works well inside one Kafka cluster.

Cross-cluster disaster recovery is where things become more interesting.

Cross-Cluster Recovery Requires Offset Translation

Suppose we have two Amazon MSK clusters:

Primary Region
    ↓
MSK Replicator
    ↓
DR Region

MSK Replicator does not copy the source partition log byte for byte. It replicates records into the target cluster and maintains mappings between source and target offsets.⁷

The same logical record can therefore have different offsets:

Primary cluster

payments
partition 3
offset 10000

and:

DR cluster

payments
partition 3
offset 9873

In general:

source offset != target offset

So simply copying a consumer group’s numerical offset into the DR cluster would be unsafe.

Amazon MSK Replicator instead synchronizes consumer group progress by translating source offsets into corresponding target offsets. It periodically records mappings between the two offset spaces, reads committed consumer group offsets from the source, translates them, and commits the translated positions into the target cluster’s __consumer_offsets topic.⁷

Conceptually:

Source                    Target

1000    ───────────────→   872
1100    ───────────────→   971
1200    ───────────────→  1074

A source checkpoint such as:

group: payment-processor
topic: payments
partition: 3
committed offset: 10001

might therefore become:

group: payment-processor
topic: payments
partition: 3
committed offset: 9874

in the DR cluster.

When the same consumer group starts against DR, it can resume near the equivalent point in the replicated stream.⁷

That is a significant DR capability. Amazon MSK is not only replicating records; it can also migrate consumer progress.

There is an important operational constraint. MSK Replicator does not overwrite synchronized offsets for a consumer group that is already active on the target cluster.⁷ The failover procedure therefore has to coordinate consumer activation with offset synchronization.

The translation is also deliberately approximate.

MSK Replicator records offset mappings periodically rather than maintaining an exact mapping for every record. AWS therefore prefers a translated position that may cause some records to be replayed rather than one that risks skipping records.⁷

For example, a source consumer may have progressed through:

10000

while the safest translated DR position corresponds to:

9997

After failover, the consumer may see the equivalents of 9997, 9998, 9999, and 10000 again.

That is at-least-once behavior, and AWS recommends that consumers tolerate duplicate processing.⁸

Recovery Position Is Not Event Identity

Now consider one of those records:

{
  "eventId": "01KABC...",
  "eventType": "PaymentCompleted",
  "paymentId": "P12345"
}

It may exist as:

Primary: payments-3 @ 10000
DR:      payments-3 @ 9873

MSK Replicator can translate between those offset spaces well enough to recover the consumer group’s position.

But neither offset intrinsically identifies the logical payment event.

That identity comes from the event contract through something such as eventId or another stable idempotency key.

This matters when the event has effects outside Kafka.

Suppose payment-processor consumes 01KABC and updates a downstream database. The consumer commits its progress, but before the corresponding checkpoint has fully propagated to DR, the primary Region fails.

The event has reached DR, but the synchronized consumer position is slightly behind.

After failover, payment-processor sees 01KABC again.

Nothing necessarily failed in Kafka or MSK Replicator. The recovery mechanism deliberately resumed from a safe position that avoided skipping records.

The application now has to answer a different question:

Have I already applied event 01KABC?

The Kafka offset cannot answer that across arbitrary external side effects.

A stable event identity or idempotency mechanism can.

That is why position and identity are complementary rather than interchangeable:

Position
    → where should I continue?

Identity
    → have I already processed this logical thing?

Kafka should not replace offsets with event identifiers. Position is what makes ordered log consumption efficient. Identity becomes important when location and execution history diverge.

Robust recovery often needs both.

What RPO = 0 Actually Means

This distinction also changes how I think about statements such as:

RPO = 0

An RPO of zero means that no state the platform considers successfully committed before the failure may be lost.

For Kafka event data, that means every record acknowledged as successfully committed by the primary platform must still be recoverable after the disaster.

This exposes an important limitation of asynchronous cross-Region replication.

Amazon MSK Replicator copies records asynchronously.⁹ A producer can therefore receive an acknowledgement from the primary cluster before that record reaches the DR cluster.

If the primary Region becomes unrecoverable during that interval, strict cross-Region event-data RPO zero cannot be guaranteed by that asynchronous replication path.

A metric such as MessageLag = 0 tells us that replication has caught up at that point in time. It does not eliminate the replication window for subsequent records.¹⁰

It is also important not to confuse three different recovery concerns:

Event missing from DR
    → data loss
    → RPO concern

Consumer checkpoint behind
    → replay
    → recovery-position concern

Business effect repeated
    → duplicate operation
    → application correctness concern

A DR cluster can contain every replicated Kafka record while its synchronized consumer position is slightly behind.

The result is replay, not necessarily data loss.

So:

Replay is not data loss. Consumer-offset lag does not by itself violate the Kafka event-data RPO.

Whether that replay is harmless depends on the consumer application’s idempotency guarantees.

What the MySQL Change Made Me Think About

The recent MySQL change did not introduce GTIDs. What caught my attention was that its newer Change Stream Applier requires GTID-based replication while explicitly leaving file-position replication channels behind.¹

That reinforces a broader recovery principle:

Recovery becomes easier when the thing being recovered has an identity independent of the location where it happened to be stored.

Kafka demonstrates why both identity and position matter.

Within one Kafka cluster, physical broker placement is hidden behind the topic-partition-offset abstraction.

Across clusters, the destination has its own log and its own offsets, so consumer progress has to be translated.

And once replay enters the picture, application-level identity becomes important because the system may need to recognize a logical event whose effects have already occurred.

For Kafka DR, I now think about recovery in layers:

Did the event survive?

Did the consumer's progress survive?

Can that progress be translated into the DR log?

If some events are replayed,
can the application recognize them?

If an event is recognized,
can its side effects safely be repeated or suppressed?

The first questions belong largely to the streaming platform.

The final questions increasingly belong to the consumer application.

Which leaves me with the question that the MySQL change originally triggered:

Does this checkpoint identify the state I processed, or merely where I should resume looking for it?

NOTE: From this post onward, I am going to tag the post where I used AI assitance for anything - review, dedupe, etc. I am honestly not waiting for humans to review and then publish. ChatGPT essentially has removed the review barrier and I am getting much better reviews than before. When I asked ChatGPT to score my intitial draft, it scored it 6/10 and suggested improvements. After improvements it says it is 8/10. I will take it. :)

These tools may not call the REST API. Sometimes that is reasonable. A recovery utility may need to work when the API is unavailable. A migration may require capabilities intentionally excluded from the public interface. An operator may need to repair partially provisioned infrastructure.

But bypassing the API must not mean bypassing the resource contract.

The governing principle is simple:

Resource integrity belongs to the resource, not to the API.

Every mechanism that creates, changes, or deletes a platform-owned resource must preserve the same resource identity, lifecycle rules, domain invariants, and required side effects. This remains true whether the operation originates from a REST endpoint, an administrative tool, an automation job, a migration, or a recovery procedure.

REST Does Not Own the Resource

REST stands for Representational State Transfer. Its uniform-interface constraints include identifying resources and manipulating them through representations.¹ A resource has an identity, commonly exposed through a URI, and a client exchanges representations of its state.²

Consider a Kafka topic managed by an event-streaming platform:

/topics/customer-events

The topic is the resource. The following JSON is one representation of its current state:

{
  "name": "customer-events",
  "partitions": 12,
  "retentionHours": 168,
  "status": "ACTIVE"
}

Changing retentionHours does not necessarily create a new resource. The resource retains its identity while its state changes.

I know many architects and engineers think of the REST endpoint as the resource unto itself, and at one point I was one of them. My reasoning was simple, the resource should have one identity, and anything that needs to interact with the resource should interact through that single identity. I once treated the API path as the resource’s canonical identity and concluded that every mutation had to pass through it. That conflated the resource with one interface used to reach it.

Of late, I have come to realize that the REST endpoint is not the resource. Nor is it a representation of the resource. The URI identifies the target resource, while the request and response payloads carry representations of its current or intended state. The API is the interaction boundary through which clients act upon that resource. The database row is not necessarily the resource either. Nor is the corresponding Kafka topic by itself. Within the platform’s control-plane model, the managed-topic resource is not necessarily identical to the physical Kafka topic. It may be a higher-level entity encompassing desired configuration, ownership, policies, provisioned infrastructure, lifecycle state, and audit history.

If the API disappears temporarily, the resource does not stop existing. If a tool avoids the API, it does not gain the right to redefine what constitutes a valid resource.

A Resource Is More Than Stored State

Suppose the platform’s topic-creation API performs the following work:

1. Validates the topic name and configuration.
2. Confirms that the caller is authorized for the owning domain.
3. Prevents duplicate creation through idempotency controls.
4. Provisions the physical topic in MSK.
5. Applies retention, encryption, and access policies.
6. Stores the platform’s metadata.
7. Publishes a lifecycle event.
8. Records an auditable history of the operation.

An administrative script that inserts a row directly into the metadata database has not necessarily created the same resource. It may only have created the appearance of one.

The platform could now report that the topic exists even though the physical topic was never provisioned. Alternatively, the physical topic might exist without the correct policies, ownership information, or audit record. Each subsystem may be locally correct while the platform resource is globally inconsistent. This is a recurring distributed-systems problem: coordinating state across multiple components introduces partial-failure and consistency concerns that cannot be reduced to one database write.³

That is the danger of treating the API implementation as the only place where integrity matters. Validation, authorization, provisioning, event publication, and auditing are not incidental HTTP behavior. They are parts of the resource’s lifecycle.

State Changes; Invariants Must Hold

The resource’s state must remain pristine in the sense that its integrity remains valid.

State is expected to change. A topic can move from PROVISIONING to ACTIVE, its retention period can be updated, and it may eventually move to DELETING and DELETED. The requirement is not to keep the original state untouched. The requirement is to ensure that every transition leaves the resource in a valid and internally consistent condition.

For a platform-managed topic, the invariants might include:

• A globally unique and stable resource identifier exists.
• The resource always has an accountable owner.
• Platform metadata corresponds to the provisioned infrastructure.
• Security and retention policies conform to platform rules.
• Lifecycle transitions follow an allowed state machine.
• Concurrent changes cannot silently overwrite each other.
• Material changes are authorized and auditable.
• Required events are published exactly as promised by the platform contract.

These invariants apply to all mutation paths. The origin of a command does not alter the definition of a valid topic. In domain-driven terms, invariants belong inside the model boundary that controls valid state transitions, not exclusively inside one delivery mechanism.⁴ Every mutation path must preserve the resource’s identity and domain invariants while honoring the applicable transition guards, audit requirements, side-effect obligations, and recovery guarantees.

The API Should Be an Adapter, Not the Domain

A robust design does not force every internal actor to communicate over HTTP merely to reuse business rules. Instead, it separates the REST adapter from the application operations that govern the resource. This follows the Ports and Adapters model, in which HTTP, batch processes, tests, and other programs can drive the application through different adapters without moving the application rules into those adapters.⁵

REST API ───────────┐
Admin CLI ──────────┤
Scheduled job ──────┼──> Application command ──> Domain rules and lifecycle
Migration utility ──┤
Recovery workflow ──┘

The REST controller translates an HTTP request into an application command. An administrative CLI may translate operator input into the same command. A scheduled reconciliation job may invoke a related command with a different authorization context. The interfaces differ, but the resource rules remain centralized and consistent.

This does not imply that every tool must execute precisely the same workflow. Operational tools sometimes need privileged operations. A migration may intentionally suppress a notification. A repair tool may reconstruct missing metadata from the physical infrastructure. A disaster-recovery process may operate while dependent systems are unavailable.

Those differences should be explicit parts of the operational contract, not accidental consequences of writing directly to storage. A privileged operation still needs defined preconditions, postconditions, authorization, auditability, and reconciliation behavior.

Direct Changes Are Break-Glass Operations

There will be situations where neither the API nor the normal application command path is usable. Direct database or infrastructure changes may then be necessary.

Such changes should be treated as break-glass operations rather than ordinary administration. The need for authorized changes, configuration-change control, and audit-record generation is also reflected in NIST’s security-control catalog ⁶. At minimum, the procedure should define:

• Who may authorize and execute the change.
• Which invariants may be temporarily violated.
• How the original and resulting states are recorded.
• How dependent state and lifecycle events are repaired.
• How the platform verifies convergence afterward.
• How the operation is made safe to retry or reverse.

The distinction is important. An emergency exception does not invalidate the resource contract. It creates an obligation to restore it.

What This Means for Platform Teams

When reviewing an operational tool, asking whether it uses the REST API is useful but insufficient. The better questions are:

1. Does it operate on the same resource identity?
2. Does it enforce the same domain invariants?
3. Does it respect the lifecycle state machine?
4. Does it reproduce or deliberately account for required side effects?
5. Does it preserve authorization, concurrency control, and auditability?
6. Can the platform detect and reconcile partial failure?

This framing also clarifies ownership. If a team owns the platform resource, it owns more than the public API. It owns the meaning and integrity of that resource across every supported way it can be changed.

The Principle

A REST API is an interaction boundary through which representations are exchanged. It is not the sole guardian of the underlying resource.

Operational tools may legitimately use a different interface. They may require elevated capabilities, alternate workflows, or direct access during exceptional circumstances. But a different mutation path does not create a different definition of the resource.

The architectural rule is therefore:

Bypassing the API is sometimes necessary. Bypassing the resource contract is not.

When this rule is applied consistently, administrative tools stop being dangerous collections of privileged shortcuts. They become deliberate participants in the platform’s resource lifecycle.

The gap between stating a principle and enforcing it matters more than the wording of the principle. A company can say it values fairness, transparency, accountability, privacy, and human oversight, and it may mean it. But without clearly assigned responsibilities, meaningful accountability, mechanisms to test behavior, and consequences for failure, those statements do not constrain what the company does.

AI makes this problem harder for software. Traditional models of professional responsibility do not fit the way software is built. A doctor can be held responsible for a diagnosis because the decision is usually attributable to the doctor. Software rarely works that way. A product decision, data choice, architecture tradeoff, model behavior, design constraint, release deadline, and legal interpretation all combine before the system reaches a user.

So when an AI system harms someone, the question is not only who wrote the code. It is also:

• Who specified the behavior?
• Who selected the training and evaluation data?
• Who accepted the product risk?
• Who decided the human review was sufficient?
• Who monitored failures after release?
• Who had the power to stop deployment?

Professionalism is not enough

One proposed answer is to treat software engineers more like licensed professionals. The idea has some merit. Software systems now shape medicine, finance, transport, hiring, speech, policing, and public administration. The people who build them often have less formal accountability than practitioners in older professions whose mistakes affect fewer people.

However, software is rarely produced by a single professional exercising independent judgment. The engineer may understand the technical risk but lack authority over the business requirement. The product leader may own the user experience but not understand what causes the model to fail. The data team may know the bias in the dataset but not own the release decision. The legal team may approve a disclaimer that is formally defensible but of little practical use to users.

Licensing individual engineers would not solve that allocation problem on its own. It may raise the baseline of competence, and it may give engineers a stronger basis for refusing reckless work, but AI responsibility has to be assigned at the system level. Accountability has to include the organization, not just the coder.

Soft law needs enforcement

That is why proposals for AI governance bodies are worth taking seriously. A global or international coordinating body for AI and robotics would not solve enforcement on its own. It could help coordinate existing forms of soft law: professional guidelines, research norms, insurance requirements, audit expectations, procurement rules, and sector specific regulation.

This matters because AI systems cross borders more easily than laws do. The GDPR gives people enforceable rights within the European Union, but many AI harms do not stay in one jurisdiction. A model may be trained in one country, deployed by a company in another, served through infrastructure in a third, and used on people who live everywhere.

Without institutions that connect principles to incentives, companies comply where enforcement exists and treat the rest as optional.

The useful governance question is therefore not what AI should value. It is which institutions can attach real costs to ignoring those values.

Collective intelligence has limits

Governance is not only about assigning responsibility after a failure. It is also about ensuring that the decision-making processes themselves remain reliable.

Collective intelligence adds another complication. Crowds can outperform individuals. Aggregated judgment can be more accurate and more stable than the judgment of any one participant. This holds in markets, elections, forecasts, open-source communities, and scientific review.

But crowds also herd. Social influence can make a group less accurate when people stop contributing independent judgment and start copying visible signals. Once that happens, the group amplifies a shared signal instead of aggregating independent ones.

This matters because many AI systems operate through feedback loops. Users rate content, models learn from users, recommender systems shape what users see, users respond to what was recommended, and the next version of the system absorbs the pattern. The crowd is part of the system rather than an external check on it.

So the question is not whether collective intelligence is good or bad. It is whether the system preserves enough independent signal for aggregation to work, and whether it can detect when influence has collapsed into herding.

Technology reveals rights

Technology also changes which rights become practically important.

The right to be forgotten, the right to public anonymity, and the right to disconnect are useful examples. These rights did not become meaningful because human interests changed. People always had interests in privacy, dignity, rest, and control over reputation. What changed was the power technology put in others’ hands.

Search engines, social media, facial recognition, cheap storage, workplace messaging, and always-on devices shifted that power. One person, company, or state can now remember, locate, classify, and interrupt another person at a scale that older social norms were not built to handle. The underlying interest existed earlier, but the corresponding duty became visible only when technology made the imbalance large enough to matter.

That is the argument for treating some rights as “revealed” by technology. These interests are not new; however, the imbalance of power that makes them urgent certainly is.

The actual work

AI governance will not be solved by choosing between ethics, law, or engineering. It needs all three.

Engineering provides evaluation, testing, audits, monitoring, incident response, system design, and evidence.. Law provides duties, remedies, procedures, and consequences. Ethics provides a way to describe what is at stake before the law addresses it.

The practical work is to connect them:

• Turn principles into concrete release criteria by requiring model cards, evaluation reports, deployment gates, rollback procedures, and incident reviews before and after release.
• Treat datasets, prompts, models, and evaluations as governed artifacts.
• Make responsibility traceable across product, engineering, data, legal, and leadership.
• Preserve independent human judgment where collective intelligence is being used.
• Recognize new technological rights before harm becomes normalized.
• Build institutions that make serious AI failures visible, attributable, and costly.

The test of AI governance is not whether an organization can write a responsible AI policy. It is whether, when the system fails, the organization can identify who was responsible for assessing the risk and whether that person had the authority to act on it. An organization that cannot answer has a policy, not governance.

References

Danny Tobey, “Software Malpractice in the Age of AI: A Guide for the Wary Tech Company”, AIES 2018.

Wendell Wallach and Gary E. Marchant, “An Agile Ethical/Legal Model for the International and National Governance of AI and Robotics”, AIES 2018.

Camelia Simoiu, Chiraag Sumanth, Alok Mysore, and Sharad Goel, “Studying the Wisdom of Crowds at Scale”, HCOMP 2019.

Jack Parker and David Danks, “How Technological Advances Can Reveal Rights”, AIES 2019.

I want to set out where I think AI genuinely helps, and where it should stay in a supporting role.

Where it helps

Hiring. Applicant tracking systems can sort, deduplicate, and surface candidates far faster than a person reading every resume. Used carefully, that saves time and removes some of the arbitrary noise in early screening. The caveat is that a model trained on past hiring decisions will also learn past hiring bias. A screening tool needs to be checked for the patterns it is reproducing, because a faster process is not automatically a fairer one.

Performance. Continuous feedback and progress tracking are easier to sustain when a system collects the signals instead of a manager reconstructing them once a year. This can make reviews less dependent on memory and recency. The risk is that the numbers a system can measure become the numbers people manage toward, even when the important part of the work is harder to quantify.

Engagement and retention. Sentiment analysis over surveys and feedback can help HR notice problems earlier, and attrition models can flag teams where people are likely to leave. This is useful as an early warning. It also shades quickly into surveillance if employees do not know what is being analyzed, so consent and transparency are not optional here.

Learning and development. Recommending training based on a person’s role, history, and stated goals is one of the lower-risk uses. It helps people find relevant material without much downside, as long as the recommendations stay suggestions rather than mandates.

Communication. Chatbots that answer routine policy and benefits questions take repetitive lookups off HR staff and give employees faster answers. This works well when the tool is scoped to retrieving information, and less well when it is asked to handle situations that need a person.

Well-being. Some tools try to infer stress or burnout from communication patterns. This is the most sensitive use on the list. It touches private information and can feel like monitoring even when the intent is supportive. If it is used at all, it should be opt-in, clearly explained, and kept away from anything that affects evaluation.

Analytics. Workforce planning, skills-gap analysis, and trend spotting are genuinely helped by aggregation. AI can show patterns across a large organization that no individual manager would see. These are inputs to decisions that people still make.

Where human judgment has to stay

AI in people management operates on data about people, and people respond to how they are measured. Once employees understand what a system rewards, some will manage toward the signal rather than the work. A metric that looked objective can quietly become a target.

The decisions here also carry moral weight. Who gets hired, promoted, flagged, or let go affects a person’s livelihood. Those outcomes should not be handed to a system that is optimizing a proxy for something it cannot directly measure. A model can inform such a decision, but it should not be the thing that makes it.

My rule is to use AI to widen what a manager can see, not to replace the manager’s judgment. Keep a person accountable for any outcome that changes someone’s job, pay, or standing, and make sure that person can explain the decision without pointing at a score.

Closing

AI is worth adopting where it reduces drudgery and improves visibility. But the judgments that matter most in people management are the ones where a person should remain responsible. The useful way to bring AI into this work is to let it handle the volume and keep humans in the decisions that carry consequences for other humans.

To have free will, a being must be able to act according to its own intent. It must not only choose, but choose in a way that is meaningfully self-determined. That makes intentionality central. A choice without intent is not really free will.

From a Hindu perspective, this distinction connects naturally to krita and karma: the choice one makes, and the action that follows from that choice. A human being is not merely pushed around by material conditions. The human being has the capacity to control desire, evaluate alternatives, and act. Those actions then shape the moral and spiritual path of the person.

That is the view of free will I find most convincing.

AI agents and choice

The harder question is whether an artificially intelligent agent can have free will.

For narrow AI, my answer is no.

A narrow AI agent can select from alternatives. It can optimize against a goal. It can make plans, call tools, respond to feedback, and produce behavior that looks increasingly independent. But that is not the same as self-determined intent.

The agent’s world is bounded by its training, context window, tools, reward structure, prompts, and permissions. It can operate within that constructed space, but it does not possess the full authorship of its own aims. It does not decide what kind of being it wants to become. It does not carry moral responsibility for its actions in the way a human being does.

We must not confuse fluent behavior with inner agency.

An AI system can say “I decided” without deciding in the human sense, or “I want” and “I believe” without desire or belief behind the words. These are useful linguistic shortcuts, but they should not mislead us into treating the system as a self-determined moral actor.

Knowledge is not enough

One reason narrow AI lacks free will is that its knowledge is incomplete and externally bounded.

To make a genuinely self-determined choice, one must be able to consider alternatives, understand consequences, and rule out paths not merely because a rule forbids them, but because one has an intentional stance toward them. Narrow AI cannot do that in the full sense. Its available alternatives are created by architecture and context. Its refusals are policy behavior, not moral renunciation.

This is where the earlier discussion of consciousness becomes relevant. If free will requires consciousness, then an AI agent would need more than inference, memory, and pattern completion. It would need perception and non-perception in a deeper sense: the ability to acquire awareness on its own, and to understand absence or non-existence with something stronger than statistical limitation.

Current narrow AI does not meet that bar.

Responsibility stays with humans

The practical conclusion is simple: do not move responsibility from humans to AI agents.

An AI agent may act. It may produce an output, make a recommendation, trigger an automation, or complete a task. But the responsibility for that action belongs to the people and institutions that designed, deployed, constrained, and benefited from the system.

Calling an AI system an “agent” is useful in engineering. It tells us the system can pursue a goal across steps. But agency in the engineering sense is not the same as free will in the moral or spiritual sense, and it helps to keep the two apart.

Humans can have free will because humans can form intent, evaluate action, and bear responsibility for karma. Narrow AI can simulate choice, but it does not own its choices. It is still an instrument operating inside limits set by human beings.

So the question worth asking is not whether AI has free will. It is whether we, the humans building and using it, are willing to exercise ours responsibly.

References

Bhagavad Gita, Chapter 7, Verse 5.

BBC Bitesize, “The nature of human life in Hinduism”.

The paper is useful because it does not treat that claim as folklore. It studies crowd performance through a large online experiment with 1,000 questions across 50 topical domains and more than 500,000 responses. The questions included numerical and categorical answers, and the domains spanned text, audio, video, and image-based tasks.

That scale matters. It lets the paper examine crowd judgment across different kinds of knowledge rather than relying on one narrow trivia-style setting.

What the paper shows

The paper reports three important findings.

First, crowds often outperform individuals. Aggregating many imperfect judgments can produce a better answer than relying on one person’s judgment.

Second, crowd performance is more consistent than individual performance. Individuals vary widely, but aggregation can smooth out some of that noise.

Third, social influence can damage crowd performance. When people see others’ answers, the group can begin to herd. Instead of adding independent information, participants may copy visible signals, and the crowd becomes less accurate.

That third finding is the most interesting part of the paper. The wisdom of crowds depends on the independence of the crowd. Once social influence becomes too strong, the mechanism that made the group useful begins to fail.

Why it matters

The paper connects directly to how modern digital systems work.

Recommendation systems, ratings, rankings, social media feeds, prediction markets, search results, and product reviews all rely on some form of aggregated human signal. These systems often assume that more participation means better information.

However, if everyone is reacting to everyone else, the signal is no longer independent. The system may look democratic while quietly amplifying early noise, popularity, status, or visibility.

That matters for AI too. AI systems increasingly learn from human feedback, user behavior, ratings, corrections, and engagement patterns. If those human signals are herded, biased, or shaped by the system itself, the model may learn the shape of the feedback loop rather than the truth of the underlying domain.

What I found especially useful

The experimental design is the paper’s biggest strength.

Using 1,000 questions across 50 domains gives the work more credibility than a small demonstration. The inclusion of different media types also helps. Crowd judgment is not one thing. Estimating a number, recognizing an image, predicting an outcome, and answering a knowledge question may all behave differently.

The paper also gives a practical warning: collective intelligence is conditional. It works best when the system preserves independent judgment and aggregates diverse evidence. It weakens when social pressure collapses that independence.

That is a good lesson for anyone designing systems around votes, likes, reviews, surveys, or human feedback.

What remains open

The paper shows that social influence can lead to herding, but it does not fully explain when that happens or why. Social influence is not always harmful. In some contexts, seeing others’ answers could help people correct mistakes or learn from better-informed participants.

The next question is therefore more specific:

• When does social influence improve crowd performance?
• When does it reduce performance?
• Which domains are most vulnerable to herding?
• Can interface design preserve independence while still allowing useful collaboration?
• How should systems detect when crowd judgment has become an echo rather than evidence?

These questions matter because many real-world systems cannot simply remove social influence. The practical challenge is to design around it.

Bottom line

“Studying the Wisdom of Crowds at Scale” is valuable because it treats collective intelligence as an empirical question rather than a slogan.

The paper shows that crowds can be powerful without being automatically wise. The crowd works when it contributes independent information. Once the system turns participants into followers of each other’s signals, it stops adding information and starts amplifying whatever was already visible.

That condition matters for AI, social platforms, and any product that depends on aggregated human judgment.

Reference

Camelia Simoiu, Chiraag Sumanth, Alok Mysore, and Sharad Goel, “Studying the Wisdom of Crowds at Scale”, HCOMP 2019.

The example is invented, even though is invented, but not far from the real ones. I have reviewed models that looked like this, and I have written a few.

The starting point

The model, as a Java record and as the OpenAPI schema that exposes it:

public record OrderData(
    String orderIdString,
    String customerEmailAddressString,
    String orderStatusValue,
    String orderTotalAmountValue,
    String orderCurrencyCodeString,
    String createdDateTimestamp,
    String updatedDateTimestamp,
    String orderNotesText,
    Boolean isOrderActiveFlag,
    String futureDiscountCode,
    Map<String, Object> extraAttributes
) {}

OrderData:
  type: object
  properties:
    orderIdString:              { type: string }
    customerEmailAddressString: { type: string }
    orderStatusValue:           { type: string }
    orderTotalAmountValue:      { type: string }
    orderCurrencyCodeString:    { type: string }
    createdDateTimestamp:       { type: string }
    updatedDateTimestamp:       { type: string }
    orderNotesText:             { type: string }
    isOrderActiveFlag:          { type: boolean }
    futureDiscountCode:         { type: string }
    extraAttributes:            { type: object }

Nothing here is wrong in the sense that a test would catch. It compiles, it serializes, it round-trips. What is wrong is that the model makes no checkable claims: Nine of its eleven fields accept any string, the names repeat themselves, and two fields – futureDiscountCode and extraAttributes exist for reasons nobody can state. Each pass below fixes one of those problems.

Pass 1 – Runs the tests: give every value a type that can reject it

Beck’s first rule is that the code passes its tests. For a model, my reading is: the type of each field should be able to test the value the field must hold. A String passes everything, so it tests nothing. The first pass replaces every stringly-typed field with a type that can reject a bad value.

public record Email(String value) {
    public Email {
        if (!value.matches("[^@\\s]+@[^@\\s]+\\.[^@\\s]+")) {
            throw new IllegalArgumentException("not an email address: " + value);
        }
    }
}

(Real email validation is more involved than one regex, and if you need it to be exact you delegate to a library. The point is not the regex; it is where the check lives. Any value of type Email has passed it, so code that receives an Email never has to re-check.)

The same treatment for the rest: an OrderId record that enforces the id format, an OrderStatus enum instead of a free-form status string, and Instant for the two timestamps, since the JDK already has a type that rejects malformed dates. The amount and the currency merge into one type, because a monetary amount without its currency is not a value you can do anything safe with:

public record Money(BigDecimal amount, Currency currency) {
    public Money {
        if (amount.scale() > currency.getDefaultFractionDigits()) {
            throw new IllegalArgumentException("too many decimal places for " + currency);
        }
    }
}

After the first pass:

public record OrderData(
    OrderId orderIdString,          // types fixed; the names are the next pass
    Email customerEmailAddressString,
    OrderStatus orderStatusValue,
    Money orderTotalAmountValue,
    Instant createdDateTimestamp,
    Instant updatedDateTimestamp,
    String orderNotesText,
    Boolean isOrderActiveFlag,
    String futureDiscountCode,
    Map<String, Object> extraAttributes
) {}

The schema gets the same pass. OpenAPI cannot run a constructor, but format, enum, pattern, and range constraints are its way of testing values before they reach your code:

orderIdString:
  type: string
  pattern: '^ord_[0-9a-z]{12}$'
customerEmailAddressString:
  type: string
  format: email
orderStatusValue:
  type: string
  enum: [pending, paid, shipped, delivered, cancelled]
orderTotalAmountValue:
  $ref: '#/components/schemas/Money'
createdDateTimestamp:
  type: string
  format: date-time

Every constraint added here is a test that runs on every request, in every environment, with no test suite involved. Alexis King’s “Parse, don’t validate” makes the general argument: once a value has parsed into a narrower type, the rest of the program can rely on it instead of re-checking it. This pass also caught a design error the original model hid: the amount and currency were two independent fields that could disagree, and Money makes that state unrepresentable.

Pass 2 – Reveals intention: say what the field is for

The types are now right but the names are still bad. The second pass renames each field to say what it is for, and Strunk’s rule 13 – omit needless words – does most of the work. For each name, strike the words that carry no information and see what is left.

• customerEmailAddressString: String repeats the type (and is now false – the type is Email). Address repeats what email already implies. What is left is customerEmail.
• createdDateTimestamp and updatedDateTimestamp: Date and Timestamp both describe the type, which the declaration already shows. What the reader needs is the event: createdAt, updatedAt.
• orderNotesText: Text describes the type. But striking it exposes a different failure: notes for whom, about what? If I cannot make a name specific, I usually have not decided what the field is for. In this system the field holds the customer’s instructions to the courier, so the honest name is deliveryInstructions.
• isOrderActiveFlag: Flag repeats the type. But after striking it, I still cannot say what active means here – not cancelled? not delivered? recently touched? A field whose meaning I cannot state is a problem for a later pass; renaming it now would not fix it.

An intention-revealing name matters more in the schema than in the Java code, because a consumer of the API cannot read the implementation. They have the field name, the type, and the description string, but the name is the one part most consumers actually read.

Pass 3 – No duplication: stop repeating the context

Every remaining name still says order, inside a type that already says it. order.orderId says order twice; so does OrderData.orderStatusValue – three times, if you count Value restating the type. Once a field lives inside Order, the context carries that word, and repeating it adds length without adding information. The Data suffix on the record name is the same duplication one level up: it describes what every record is. In Vibe Coding and the Baby Genius Problem I made the same case against Request and Response suffixes on API models.

This is Russ Cox’s getParametersAsNamedValuePairArray point at the field level – the only interesting word in that name is parameters, and the only interesting word in orderStatusValue is status.

The pass also catches duplication that is not in the names. isOrderActiveFlag turned out to mean “status is neither cancelled nor delivered” – it restates status, as data. Since the value is derivable from status, I delete the field rather than rename it; if the convenience matters, an isActive() method on the record can compute it.

public record Order(
    OrderId id,
    Email customerEmail,
    OrderStatus status,
    Money total,
    Instant createdAt,
    Instant updatedAt,
    String deliveryInstructions,
    String futureDiscountCode,
    Map<String, Object> extraAttributes
) {}

Pass 4 – Fewest elements: delete what has no reason to exist

Two fields remain that no current feature reads.

futureDiscountCode was added because a discounts feature is expected someday. When that feature arrives it will have real requirements, and this field – named, typed, and shaped before any of them were known – is unlikely to match. Until then it is surface area: it appears in generated clients, consumers store it, and when the real feature ships it will have to be deprecated in favour of whatever is actually needed.

extraAttributes is a bigger problem, because it weakens the whole schema rather than just adding an unused field. A Map<String, Object> field says that anything may appear here, with any type, and none of it is documented. Whatever goes into that map is exactly the data that should have been a named, typed field, and once consumers start passing values through it you can never remove it. The only cheap time to delete it is while it is still empty.

The final model:

public record Order(
    OrderId id,
    Email customerEmail,
    OrderStatus status,
    Money total,
    Instant createdAt,
    Instant updatedAt,
    String deliveryInstructions
) {}

Order:
  type: object
  required: [id, customerEmail, status, total, createdAt, updatedAt]
  properties:
    id:
      type: string
      pattern: '^ord_[0-9a-z]{12}$'
    customerEmail:
      type: string
      format: email
    status:
      type: string
      enum: [pending, paid, shipped, delivered, cancelled]
    total:
      $ref: '#/components/schemas/Money'
    createdAt:
      type: string
      format: date-time
    updatedAt:
      type: string
      format: date-time
    deliveryInstructions:
      type: string
      maxLength: 500

Seven fields instead of eleven, every one of them constrained, and every name down to the words that carry information. Cox’s rule about length and scope explains why the final names can be this short: id is unambiguous precisely because it is read inside Order, where the context supplies the rest. The same field flattened into a log line or an analytics event should be orderId again, because there the context is gone and the name has to carry it.

Trade-offs

The refactor above is presented as if it were free. It is not, and it helps to know the costs before applying it.

Wrapper types are ceremony, and serialization notices. Every Email-style type needs a Jackson @JsonValue and @JsonCreator (or the equivalent in your framework) so it serializes as a plain string rather than a nested object. Records have made the class definitions nearly free, but the serialization plumbing is a real, one-time cost per type, and on a small internal tool it may not pay for itself.

Some strings are honestly strings. deliveryInstructions stayed a String because free text is genuinely a string: there is no constraint a DeliveryInstructions type could test beyond a length cap, and the schema’s maxLength already covers that. Wrapping it would add a type without adding a check. The test I use: if the constructor would be empty, the wrapper adds nothing.

Renaming a published field is a breaking change. Everything in this post is cheap at design time and expensive after the first consumer integrates. Once orderIdString is in production and has seen traffic, fixing it means versioning the API or carrying both names through a deprecation cycle. This is the strongest argument for doing the naming work before v1 ships rather than after.

Short names depend on their context surviving. id is right inside the schema and wrong in a CSV export. If a name will be read where its enclosing context is stripped away, it has to carry the context itself. Cox’s rule decides both cases; it just gives different answers.

A consistent team convention beats my preference. If the codebase I am in writes orderId on every entity, I write orderId. A reader who can predict every name in the system is better off than one who has to learn which files follow which philosophy.

Sources

• Kent Beck’s four rules of simple design, as summarized by Martin Fowler
• The Elements of Style, rule 13: omit needless words, Strunk
• Notes on naming, Russ Cox
• Parse, don’t validate, Alexis King
• My naming philosophy, the short version of this post
• Vibe Coding and the Baby Genius Problem, an earlier post of mine on making naming conventions enforceable

At its simplest, the agent only needs to choose a legal card. It can match color, match value, play a wild card, draw cards, skip turns, and remember to say “UNO” before its last card. That is enough to build a production system: a set of if conditions connected to actions.

For example:

• If the discard card is a skip or reverse card, skip the turn when required.
• If the agent has a card matching the current color, play it.
• If the agent has no color match but has a card with the same value, play that.
• If no ordinary card is playable, consider a wild card.
• If a wild card is played, choose the color that gives the agent the strongest remaining hand.
• If no legal play exists, draw.

Because the behavior is explicit, you can inspect the rules and explain why every move was made. That explainability is one of the main strengths of production systems. Every action can be traced back to the rule that fired.

Here is a small example of how such an agent behaves.

Suppose the visible discard card is yellow 8, and the agent’s hand contains red numbered cards, two blue 8 cards, a blue 3, a green draw-two card, and a wild draw-four card. The agent cannot match yellow, but it can match the value. So it plays a blue 8. This is a legal move, but it is not necessarily strategic. It simply follows the first useful rule that applies.

Now suppose the next visible card is a blue skip. If the skip was not produced by the agent’s own previous move, the agent loses the turn and records that the skip effect has been consumed. That small bit of state matters. Without it, the agent might keep reacting to the same skip card as if it were a new event.

Near the end of the hand, the agent may have only the green draw-two and wild draw-four left. If the discard is red 9, it has no color match, no value match, and no ordinary wild card. So it plays wild draw-four, chooses the color that best matches its remaining hand, and says “UNO” because it is down to one card. If the chosen color is green, the final green draw-two becomes playable on the next turn.

The agent simply follows a priority order:

• match color if possible
• otherwise match value
• otherwise use a wild option
• choose the next color based on the remaining hand
• maintain enough state to avoid misreading skip and draw effects

That is enough to play a legal sequence. It is not enough to be a strong player because a legal move is not always a good move.

A simple UNO production system can play the game, but it does not necessarily play to win. It can follow the rules while still making weak choices. It may play a special card too early, hold the wrong color too long, or fail to account for an opponent’s likely hand.

A rule-based agent can be competent at legality without being competent at strategy. To become stronger, the agent needs more than local rules. It needs memory and search:

• Remember previous moves.
• Track the opponent’s visible moves.
• Remember which colors and values have appeared.
• Estimate what cards the opponent might still hold.
• Search across possible future states.
• Compare legal moves by expected advantage, not only immediate playability.

Once those capabilities are added, the agent changes character. It is no longer merely asking “what can I play?” It begins asking “what should I play, given what might happen next?”

This small example maps cleanly to larger AI systems. Many production systems are, at their core, rule-based agents. They approve, reject, route, escalate, notify, retry, block, or transform based on explicit conditions. That design is often the right one. It is inspectable, debuggable, and easy to constrain.

The limitation appears when success depends on strategy rather than legality. A production system can explain every decision while still making poor long-term choices because it lacks planning, opponent modeling, and learning from experience. That is not a flaw in production systems; it reflects the kinds of problems they were designed to solve. UNO makes the distinction clear: playing a legal card requires rules, but playing well requires knowledge, memory, and search.

The paper’s central idea is the “revealed right”: a right that becomes meaningful only in a particular technological context. The right is not invented from nothing. The underlying human interest may already exist. What changes is the technology. It shifts the balance of power enough to reveal a corresponding duty.

That framing is helpful because it avoids two weak positions. It does not pretend that every new technological discomfort automatically creates a right. It also does not pretend that rights are fixed forever in the form they had before modern technology changed the world.

The framework

The paper describes a pattern:

Parties have important but conflicting interests. A technology gives one party more causal power to advance its interests, while weakening another party’s ability to protect theirs. When that imbalance becomes large enough, a right may be revealed.

This is a practical framework because it connects rights to changes in power rather than to technological novelty.

We usually expect power to track the ethical weight of an interest. If one person’s interest is much more important, we expect the social or legal system to protect that interest more strongly. Technology can break that expectation. It can give enormous practical power to a party whose interest is actually less weighty.

That is where rights become newly urgent.

Examples

The paper discusses rights such as the right to public anonymity, the right to be forgotten, and the right to disconnect.

These examples make sense because the underlying interests are old, but the scale of intrusion is new.

People have always cared about privacy, reputation, rest, and control over their social presence. But search engines, social media, cheap storage, facial recognition, smartphones, and workplace messaging changed the balance. They made it inexpensive and scalable for companies, governments, employers, and strangers to remember, locate, classify, interrupt, or expose people in ways that older norms did not fully anticipate.

The result is not simply “technology is bad” but that it can disturb the balance between interests and power. When that happens, rights language becomes a way to restore the balance.

What I found useful

The paper’s strongest contribution is that it gives a framework for evaluating new rights without treating them as arbitrary.

The question is not merely whether someone wants protection. The question is whether a technology has changed the distribution of causal power between parties, and therefore the duties they owe one another.

This is especially useful for AI. AI systems can change power relationships quickly:

• Employers can monitor workers more closely.
• Platforms can classify users cheaply.
• Governments can monitor and process large populations at scale.
• Companies can infer sensitive traits from ordinary behavior.
• Automated systems can make decisions that are difficult for individuals to understand or contest.

In each case, the ethical question is not only whether the technology works. It is whether the technology has created a new imbalance that requires a corresponding duty.

What remains open

The paper deliberately focuses on when rights become visible, not on how institutions should recognize, enforce, or balance those rights.

Some rights may need formal legal protection. Others may be better handled through regulation, product design, professional norms, workplace policy, or social expectations. The paper gives a strong account of how a right becomes visible, but less detail on how that right should be honored in practice.

That leaves several important questions:

• Which revealed rights should become law?
• Which should become product obligations?
• Which can be handled through social norms?
• Who owes the duty: governments, companies, employers, platforms, or individuals?
• What remedies should exist when the right is violated?
• How should technology companies build revealed rights into product, legal, and marketing strategy?

Those questions are where the theory has to become governance.

Bottom line

“How Technological Advances Can Reveal Rights” is valuable because it explains why technology can make old interests newly urgent.

The paper does not argue that every technological change creates a new right. It argues that technology can shift causal power so dramatically that a previously latent right, and the corresponding duty, become visible. The novelty of the paper is not that it argues for new rights, but that it explains when technological change justifies recognizing them.

That is a strong way to think about AI-era rights. It asks us to look beyond the interests people hold and examine how technology has changed the distribution of power over those interests and what duties should now exist because of that change.

Reference

Jack Parker and David Danks, “How Technological Advances Can Reveal Rights”, AIES 2019.

And yet, collectively, we are remarkably bad at behaving.

People still speed. People still jump red lights. People still drink and drive. We continue to fight over race, religion, nationality and ideology. We lie, cheat, steal and kill. Every generation inherits thousands of years of human history, yet somehow manages to repeat many of the same mistakes.

Maybe learning simply takes time. Maybe every generation becomes a little wiser than the previous one. History, however, does not inspire much confidence. It is full of wars that should have convinced us not to start new ones. It is full of financial crises that should have made us more careful. It is full of genocides that should have made future genocides unthinkable. Yet here we are.

There is an old saying that history repeats itself.

Of course it does.

If we struggle to learn from our own mistakes, expecting us to learn from the mistakes of people who lived centuries ago may simply be asking too much.

The psychologists have spent decades studying this very question. I expected to find evidence that failure is our greatest teacher. Instead, I found that the reality is much less encouraging.

We learn much less from failure than we think

In Not Learning From Failure—The Greatest Failure of All, Lauren Eskreis-Winkler and Ayelet Fishbach found that people often learn less from failure than from success, even when failure provides exactly the same information.¹

The researchers identify two reasons. The first is emotional. Failure hurts. Rather than examining it, people often avoid thinking about it because it threatens their self-image.

The second is cognitive. Failure tells us what did not work, but it rarely tells us what will. Figuring out the lesson requires considerably more effort than repeating something that already succeeded.

We like to say that failure is the best teacher. Psychology suggests that it often isn’t — because we simply refuse to listen.¹

Institutions remember what people forget

Research by Dahlin, Chuang and Roulet points to an interesting difference between individuals and organizations.²

Individuals often fail to learn from failure.

Organizations can do much better, but only when they deliberately preserve the lesson. Accidents become investigations. Investigations become reports. Reports become procedures. Procedures become standards.

People forget.

Institutions remember.

That is one reason aviation has become safer over time. Every major accident leaves behind documentation that changes how future pilots are trained. Drivers see accidents every day, but very few permanently change how they drive because someone else crashed.²

The institution learns. The individuals often do not.

Civilization accumulates knowledge

Joseph Henrich argues that civilization advances because knowledge accumulates across generations rather than because individuals are exceptionally intelligent.³

Nobody has to rediscover calculus, electricity or germ theory. We inherit them. Every generation starts where the previous one finished.

That is how science progresses.

That is how engineering progresses.

Technical knowledge accumulates because we know how to preserve it.

Behavioral wisdom does not seem to work the same way.

We do not learn equally from everyone

Research on cultural evolution shows that humans copy some people more readily than others. We imitate people who are successful, respected, prestigious or part of our own group.⁴

Sometimes those people deserve to be copied. Sometimes they do not.

Good behavior spreads this way. Bad behavior does too.

Knowing something is not enough

Psychologists have known for a long time that knowledge alone rarely changes behavior.⁵

Most people know smoking causes cancer.

Most people know speeding is dangerous.

Most people know excessive drinking impairs judgment.

Most people know obesity increases health risks.

Yet millions continue doing all of these things.

Behavior is shaped not only by knowledge, but also by habits, incentives, emotions, identity and social pressure.⁵

Knowing better does not necessarily mean doing better.

We think everyone else is the problem

There is a well-known finding in psychology called the Better-than-Average Effect. Most drivers believe they are above-average drivers, even though that is statistically impossible.⁶

If I already believe I am safer than everyone else, why would I change after seeing an accident? I assume the lesson is for someone else.

Our brains were built for a different world

Evolutionary psychologists argue that the human mind evolved in small groups where feedback was immediate and consequences were local.⁷

Today we live with highways, financial markets, nuclear weapons, social media and artificial intelligence.

The world changed much faster than the human mind.

So why does history repeat itself?

Historians correctly point out that history never repeats exactly.

The people change.

The technologies change.

The governments change.

What changes much more slowly is human psychology. Every generation believes it is different. Every generation believes it is smarter than the last. Every generation believes the warnings apply to someone else. That is probably why history keeps rhyming.

The real problem

We have become exceptionally good at accumulating technical knowledge. We preserve it in books, scientific journals, engineering standards, universities and institutions. Every generation inherits that knowledge and adds a little more.

We have never built an equally effective way to accumulate behavioral wisdom.

Every generation still has to learn patience, prejudice, greed, empathy, cooperation and restraint almost from scratch. We continue making many of the same mistakes because the lessons never become part of a shared, reliable memory in the way scientific knowledge does.

Perhaps that is the real difference.

Knowledge accumulates. Behavioral wisdom barely does.

References

The core proposal is a Governance Coordinating Committee, or GCC, that would help coordinate ethical, legal, technical, and institutional responses to AI and robotics. The paper is not simply asking for another principles document. It is trying to describe an institutional mechanism for turning soft law into something more durable.

AI governance already has many declarations, principles, and high-level frameworks. The problem is not a shortage of ethical vocabulary. The problem is that fast-moving technology often outruns the institutions that are supposed to discipline it.

The paper’s strongest argument is that AI and robotics governance cannot be left entirely to one actor.

Governments move slowly and are constrained by jurisdiction. Engineers understand technical feasibility but do not always have authority or understanding of the consequences. Ethicists can clarify values and harms, but ethical analysis alone does not create compliance. Courts need framework in terms of laws, rules and precedents to make a judgment.

The proposed GCC is meant to sit across these groups and coordinate governance through multiple channels:

• Government regulation
• Engineering standards
• Ethical guidance
• Insurance requirements
• Academic publication norms
• Grant funding conditions
• Judicial interpretation
• Organizational compliance programs

That is a realistic way to think about AI governance. In practice, behavior changes when incentives converge from multiple directions. If insurers, journals, funders, regulators, and courts all begin expecting similar safety and transparency practices, companies have a stronger reason to treat those practices as real requirements instead of optional ethics theater.

Soft law is often criticized because it lacks direct enforcement. That criticism is fair, but then why does it matter?

In fast-moving areas such as AI, soft law can do work that formal law cannot do quickly enough. It can establish norms, define expected practice, create audit vocabulary, and give courts or regulators a reference point when formal disputes arise.

The paper’s useful insight is that soft law becomes stronger when institutions coordinate around it. A guideline from one professional group is easy to ignore. The same guideline becomes harder to ignore if it affects insurance, publication, procurement, grant funding, and litigation risk.

This is especially relevant for algorithmic transparency. Some AI systems may be explainable enough for ordinary review. Others may be opaque in ways that require different safeguards. A governance body could help define when opacity is acceptable, what testing is required before deployment, and which contexts should reject opaque systems entirely.

I agree with the paper’s broad direction. AI and robotics need governance that is international, adaptive, and institutionally connected.

The GDPR is a useful comparison. It gives enforceable rights within the European Union and has shaped global corporate behavior beyond Europe. But AI systems are not always contained within one legal jurisdiction. Models, data, users, vendors, and infrastructure can be spread across countries. A coordinating body could help reduce the gap between regional laws and global deployment.

The paper is also right that engineering and ethics need to be connected. Ethical principles are weak if they never become technical requirements. Engineering practices are dangerous if they optimize only for capability and ignore rights, safety, and human consequences.

However, the proposal leaves hard implementation questions open.

The biggest question is authority. Would the GCC merely recommend standards, or would it have some route to enforcement? If enforcement depends on other institutions, how would those institutions be coordinated? How would the GCC avoid becoming another advisory body whose output companies cite without changing behavior?

There are also questions about legitimacy:

• Who appoints the GCC?
• How are affected communities represented?
• How does it avoid capture by large technology companies?
• How does it handle disagreement between countries with different values and legal systems?
• How does it connect to existing laws such as GDPR?
• How would it incorporate human rights declarations such as the Toronto Declaration into operational requirements?
• How GCC will operate internationally?
• How will GCC attain autonomy and at the same time have authority delegated by the government of different nations?

The paper’s GCC proposal may or may not be the exact institution the world needs, but the underlying point is sound: AI principles require coordinated enforcement pathways. Without that, responsible AI stays mostly a matter of language.

Writing down what AI should respect is the easy part. Now, the harder work will be to build the institutions that make those commitments difficult to ignore.

Reference

Wendell Wallach and Gary E. Marchant, “An Agile Ethical/Legal Model for the International and National Governance of AI and Robotics”, AIES 2018.

A neuron does not think. An ant does not plan the long-term fate of a colony. But a brain produces cognition through the interaction of neurons, and an ant colony produces coordinated behavior through the interaction of ants. In both cases, the interesting property lives at the level of the system, not the individual unit.

That does not prove that an anthill is conscious. It does, however, make the question harder to dismiss.

The analogy

At the simplest level, the mapping looks like this:

• A neuron maps to an individual ant.
• A brain maps to an ant colony.
• Neurotransmitters and dendrites map, roughly, to the signals ants exchange through antennae, movement, scent, and local interaction.
• Cognition maps to the organized behavior that emerges from many small interactions.

The analogy is not perfect. A brain and a colony are built differently, operate at different speeds, and solve different classes of problems. But the analogy is useful because it forces a discipline on the question. If we are willing to talk about cognition as something that emerges from many non-conscious parts in the brain, we should at least ask what kind of intelligence emerges from many simple agents in a colony.

A Vedantic test

In an earlier post I used the six pramanas from Advaita Vedanta as a way to think about intelligence and artificial intelligence. They are not a laboratory test in the modern scientific sense. They are a philosophical framework for asking how knowledge is acquired and justified.

Applied to an anthill, they give us a useful checklist.

Pratyaksha, or perception:
The colony can sense its surroundings through the distributed activity of individual ants. No single ant perceives the whole environment, but the colony can still respond to food, threat, nest quality, temperature, and other conditions.

Anumana, or inference:
The colony can turn local signals into collective decisions. A change in behavior follows from accumulated evidence, even if no individual ant is performing explicit reasoning in the human sense.

Upamana, or comparison and analogy:
Colonies can compare alternatives. Nest selection is a good example: scouts evaluate candidate sites, and the colony eventually converges on one option over another.

Arthapatti, or postulation:
The colony can behave as if it is evaluating implications. If one nest site has better darkness but a worse entrance, and another has the reverse, the colony’s final movement reflects a tradeoff across conditions.

Anupalabdhi, or non-perception:
The colony can incorporate absence. If a path stops producing food or a site fails to attract enough confirming activity, the colony’s behavior changes. This is not a formal proof of non-existence, but it is a practical response to missing evidence.

Shabda, or word/testimony:
A colony has memory beyond the life of an individual ant. Its trails, nest structure, and learned patterns can persist across generations. In that sense, the colony can rely on information that no single living ant originated alone.

Under this framework, an anthill does surprisingly well.

But is that consciousness?

The checklist shows that an anthill can satisfy several conditions we associate with intelligence: perception, inference, comparison, tradeoff, response to absence, and retained knowledge. That makes the anthill a strong example of collective intelligence.

But consciousness is a heavier claim than intelligence.

A system can process information without having inner experience. A market can aggregate signals. A city can route traffic. A software organization can remember habits that no employee wrote down (tribal knowledge). These systems can be intelligent in a distributed sense without obviously having a point of view.

An anthill behaves like a system with distributed cognition. It acquires information, compares alternatives, responds to missing signals, and preserves useful patterns across time. If consciousness is defined only in terms of knowledge acquisition and coordinated response, then the colony begins to qualify. If consciousness requires subjective experience, then the analogy becomes suggestive but not decisive.

Why this matters for AI

Modern AI systems are increasingly built as networks of smaller processes: models, tools, prompts, retrieval systems, evaluators, schedulers, memory stores, human feedback, and organizational review. Like an anthill, the behavior of the system is not located cleanly in any single part.

Once a distributed system begins producing impressive behavior, people start reaching for words like intelligence, agency, and consciousness. Sometimes those words clarify. Often they hide the real engineering question.

Asking whether the system feels conscious is not very useful, because we do not know how to answer it well. It is more practical to ask what the system actually does:

• What does the system perceive?
• What can it infer?
• What alternatives can it compare?
• What tradeoffs can it evaluate?
• What absences can it detect?
• What past knowledge does it rely on?

Those questions are practical. They can be tested. They also keep us from giving a system metaphysical credit[^1] for behavior that may be explainable through coordination, feedback, and memory.

Complex behavior does not require a tiny human sitting inside the system, making central decisions. Intelligence can emerge from interaction. But the existence of emergence does not automatically settle the question of consciousness, and it is worth keeping those two questions apart.

References

Bob Holmes, “The Mind of an Anthill”, Knowable Magazine, 2018.

Deborah M. Gordon, “Your brain has more in common with an ant colony than you realised”, World Economic Forum, 2019.

Eliot Deutsch and Rohit Dalvi, The Essential Vedanta: A New Source Book of Advaita Vedanta, 2004.

Notes

[^1] In this context, “metaphysical credit” means attributing consciousness or subjective experience to a system just because it produces impressive behavior. It’s the leap from “this system does something smart” to “this system must be aware of what it’s doing.” My point in this post is that complex coordinated behavior, such as an anthill solving problems, an AI generating text, etc. can be fully explained by local interactions, feedback, and memory without invoking inner experience. Giving the system “metaphysical credit” is treating emergence as evidence of consciousness, when it might just be coordination.

There are models that can paint a picture. Image generators such as Nano Banana return finished pixels. But a finished picture can only be accepted, rejected, or regenerated. A language model produces text, and an SVG stores a picture as text. The model drew the banner by writing out the file the source of which I could edit.

SVG is not unusual in this. Mermaid describes flowcharts in a few lines of a small language. Graphviz does the same for graphs with DOT. PlantUML covers the standard software diagrams like sequence, class, and component. Each of these formats has an interesting story behind it. SVG is XML because it was designed as a web standard, and browsers render markup. DOT is text because Graphviz was built to lay out graphs that programs generate. Mermaid and PlantUML are text so that a diagram can live next to the prose and code it describes. There is one more interesting thing about all of them using text to describe a diagram. A diagram that is text can be diffed, reviewed, and committed like any other source file.

None of these formats was designed with a language model in mind. Yet they are exactly what a language model needs. Here is a flowchart, written as text:

flowchart LR
  client[Client] --> gw[API Gateway]
  gw --> ent[Entitlements]
  ent -.denied.-> client

The renderer this blog already uses turns those four lines into a picture:

flowchart LR
  client[Client] --> gw[API Gateway]
  gw --> ent[Entitlements]
  ent -.denied.-> client

When the design changes, I just need to edit four lines instead of redrawing anything. A model can write those four lines as easily as I can, because they sit close to how the picture would be described in words. The same holds for DOT and PlantUML: small vocabularies, named shapes, little room to go wrong.

There is a limit, and it is the same property seen from the other side. Writing a picture as text works only where the picture is simple enough to be written as text. Icons, badges, flowcharts, architecture diagrams — things built from clean primitives — sit comfortably inside it. A face or an animal does not. The model can still emit a path with fifty control points, but at that point the source is no longer something a person can read or fix, and the reason for wanting text is gone.

If diagrams had stayed pixels, drawing one would belong to the image models, and the result would be a flat picture with nothing to diff and nothing to edit. Because these formats store the picture as text, the model writes the source, and the renderers that were already there turn it into a picture. The formats were built for browsers, for graph layout, for documentation. Nobody planned the second use.

A    U+0041
a    U+0061
अ    U+0905
क    U+0915

The explanation is correct to an extent; however it is not precise. It suggests that Unicode is a very large ASCII table: ASCII has a number for A, so Unicode has a number for अ; ASCII has a number for B, so Unicode has a number for क. From there it is a short step to assuming that every Hindi character on the screen has its own Unicode number. It does not and looking at how Devanagari⁴ actually works in Unicode gives us a way to look into the workings of Unicode.

Start with क

The Devanagari letter क has the code point:

U+0915    DEVANAGARI LETTER KA

So far the ASCII model holds. One letter, one number.

Now consider कि.

A Hindi reader sees this as a single syllabic unit, and the vowel sign ि⁵ appears visually to the left of the consonant. It would be reasonable to assume the computer stores it in that order. It does not. The stored sequence is:

क    U+0915    DEVANAGARI LETTER KA
ि    U+093F    DEVANAGARI VOWEL SIGN I

That is, U+0915 U+093F – consonant first, vowel sign second. If you have ever typed in Hindi on your iPhone, that is the order you type as well.

Something happened between the sequence in memory and the pixels on the display. That something is text shaping⁶.

Unicode describes logical text, not pixels

Unicode encodes characters and their logical order. It does not specify the shapes that end up on screen. The path from one to the other runs roughly like this:

code points → encoded bytes → shaping engine → glyph selection → pixels

For Latin text most of this machinery is easy to overlook. cat is stored as c, a, t and displayed as c a t. Stored characters and displayed glyphs⁷ look one-to-one.

Devanagari breaks that assumption immediately.

क्ष

The conjunct⁸ क्ष looks like one character. Unicode does not encode it as one. It is:

क    U+0915    DEVANAGARI LETTER KA
्    U+094D    DEVANAGARI SIGN VIRAMA
ष    U+0937    DEVANAGARI LETTER SSA

The virama⁹ is the key element. It suppresses the inherent a vowel¹⁰ that a Devanagari consonant letter carries by default, which is what allows two consonants to join. When the shaping engine sees consonant + virama + consonant, it applies the font’s conjunct rules and may produce a single ligature glyph, or a half-form¹¹ of the first consonant followed by the full second consonant, depending on the font. Both are correct renderings of the same three code points.

So one thing on screen, three code points underneath.

Five terms that are not synonyms

Once you accept that, the vocabulary has to get more precise. Five terms get used interchangeably and should not be:

Code point. A number in the Unicode code space, from U+0000 to U+10FFFF. U+0915 is a code point.

Character. An abstract textual element. क is a character, represented by the code point U+0915. Unicode itself uses “character” loosely, which is part of the problem.

Code unit. The unit of the encoding form you chose. UTF-8¹² has 8-bit code units; a Devanagari code point takes three of them. UTF-16 has 16-bit code units, and nearly¹³ every Devanagari code point fits in one, because the main block sits in the Basic Multilingual Plane¹⁴. UTF-32 has 32-bit code units, one per code point.

Grapheme cluster. A user-perceived character, defined by the segmentation rules in UAX #29¹⁵. This is the closest thing to what a reader would call “one character.”

Glyph. A shape in a font. Glyphs are what a font actually contains, and the mapping from characters to glyphs is many-to-many.

Almost every bug in this area comes from a program using one of these when it meant another.

Counting the same string five ways

Here are the three examples measured against each definition. All Devanagari code points encode as three bytes in UTF-8 and one code unit in UTF-16.

नमस्ते decomposes as:

न    U+0928    DEVANAGARI LETTER NA
म    U+092E    DEVANAGARI LETTER MA
स    U+0938    DEVANAGARI LETTER SA
्    U+094D    DEVANAGARI SIGN VIRAMA
त    U+0924    DEVANAGARI LETTER TA
े    U+0947    DEVANAGARI VOWEL SIGN E

Six code points. A Hindi reader would say three units: न, म, स्ते. Depending on your runtime, a grapheme segmenter will tell you three or four.

The cluster count depends on your runtime, not your code

Before Unicode 15.1, the extended grapheme cluster rules broke after a virama. क्ष counted as two clusters, नमस्ते as four. Unicode 15.1 added rule GB9c¹⁶, which uses the new Indic_Conjunct_Break¹⁷ property to hold consonant + linker + consonant together. Devanagari virama has InCB=Linker, so under 15.1 क्ष is one cluster.

This is not hypothetical. Here is java.text.BreakIterator¹⁸ on JDK 21:

हिन्दी     [हि][न्][दी]              3 clusters
क्षत्रिय   [क्][ष][त्][रि][य]        5 clusters
नमस्ते     [न][म][स्][ते]            4 clusters
कि         [कि]                      1 cluster
क्ष        [क्][ष]                   2 clusters

Every one of those is the pre-15.1 answer. The conjuncts split at the virama.

There are two things going on here. First, JDK 21 ships Unicode 15.0 character data, not 15.1 – you can confirm this without documentation by probing a marker code point:

Character.isDefined(0x11B00)   // true  -- Devanagari Extended-A, Unicode 15.0
Character.isDefined(0x2EBF0)   // false -- CJK Ext-I, Unicode 15.1

Second, and more important, java.text.BreakIterator carries its own segmentation rule data that is largely independent of the character tables. Upgrading the JDK does not reliably get you GB9c. For 15.1 behaviour in Java you need ICU4J’s¹⁹ com.ibm.icu.text.BreakIterator.

So the pre-15.1 column above is measured. The 15.1+ column is what the current rules specify. Which one your program produces is a property of the library you happened to link.

What length actually returns

The draft note on this post suggested that Java, JavaScript and Python disagree about the length of Hindi text. They mostly do not, and the reason is worth understanding.

For नमस्ते:

"नमस्ते".length()                      // 6   UTF-16 code units
"नमस्ते".codePointCount(0, 6)          // 6   code points
"नमस्ते".getBytes(UTF_8).length        // 18  bytes

"नमस्ते".length                        // 6   UTF-16 code units
[..."नमस्ते"].length                   // 6   code points

len("नमस्ते")                          # 6   code points
len("नमस्ते".encode())                 # 18  bytes

len("नमस्ते")                          // 18  bytes
utf8.RuneCountInString("नमस्ते")       // 6   code points

"नमस्ते".len()                         // 18  bytes
"नमस्ते".chars().count()               // 6   code points

Java and JavaScript store strings as UTF-16 and report code units. Python stores code points and reports code points. Go and Rust store UTF-8 and report bytes. For Devanagari, the first three agree at 6, because every Devanagari code point is a single UTF-16 code unit.

The divergence between Java/JavaScript and Python shows up outside the BMP. For the single emoji 😀 (U+1F600), Java and JavaScript report 2, because it needs a surrogate pair; Python reports 1; Go and Rust report 4 bytes.

The more useful observation is that none of these numbers is 3, which is the number a Hindi reader would give for नमस्ते. Getting closer requires explicit grapheme segmentation:

BreakIterator.getCharacterInstance()   // 4 on JDK 21 -- see above

new Intl.Segmenter("hi", { granularity: "grapheme" })

import regex
regex.findall(r"\X", "नमस्ते")

// unicode-segmentation crate
"नमस्ते".graphemes(true).count()

Even these depend on the ICU or Unicode data version, per the GB9c caveat above.

Normalization: two ways to write the same letter

Devanagari also has a normalization problem. The letter क़ (qa) can be written two ways:

U+0958                    DEVANAGARI LETTER QA
U+0915 U+093C             KA + DEVANAGARI SIGN NUKTA

These are canonically equivalent²⁰. They must be treated as the same text by any conforming process, and they usually render identically. But they are different byte sequences, so equals, ==, hash lookups, and database unique constraints will treat them as different unless the text is normalized first.

There is a trap here. U+0958 through U+095F are on the Unicode composition exclusion list²¹. NFC²² does not recompose them. Normalizing either form to NFC gives you U+0915 U+093C, the decomposed one. If you assumed NFC always produces the shortest sequence, this is a counterexample.

Normalization also reorders combining marks by canonical combining class²³. Nukta²⁴ has ccc=7, virama has ccc=9, so a nukta typed after a virama gets reordered ahead of it:

as typed : U+0915 U+094D U+093C     क + virama + nukta
NFC      : U+0915 U+093C U+094D     क + nukta + virama

Two sequences that differ only in the order the user typed them collapse to one form.

The practical rule: normalize at your system boundaries – on input, before storage, before comparison – and pick one form, usually NFC.

Sorting is a separate problem

Normalization fixes equality. It does not fix order, and binary comparison²⁵ gets Devanagari order wrong in a way that is easy to miss.

The reason is where the nukta letters sit in the block. क is U+0915, ख is U+0916, ह is U+0939. But क़ has its own precomposed code point at U+0958, past the end of the consonant range. Sort by code point and क़ lands after every basic consonant, including ह.

                              compareTo   Collator(hi)
क  vs ख                              -1            -1
क  vs का                             -1            -1
का vs कि                             -1            -1
क़  vs ख                              +1            -1
क़  vs क+़                            +1             0
ड़  vs ढ                              +1            -1

The first three pairs agree, which is why this bug survives casual testing. Compare a few basic letters and binary order looks fine. The nukta letters are where it breaks, and the last row is worse than wrong order: two canonically equivalent spellings of the same letter compare as unequal.

Sorting a word list makes it concrete. The same word क़लम, written once with U+0958 and once with U+0915 U+093C:

String.compareTo        Collator(hi)
  कमल                     कमल
  कल                      कल
  क़लम   (U+0915…)         क़लम   (U+0958…)
  खत                      क़लम   (U+0915…)
  गाना                      खत
  क़लम   (U+0958…)         गाना

Binary order scatters the two spellings to positions three and six with unrelated words between them. The collator files them adjacent, both immediately after क.

One trap inside the fix. Collator.getInstance(...).getDecomposition() returns NO_DECOMPOSITION by default. The Hindi collator handles U+0958 anyway, because the JDK ships real hi rules – Collator.getInstance(Locale.ROOT) returns +1 for that pair rather than 0. Do not rely on the default. Set CANONICAL_DECOMPOSITION explicitly, and pick the locale deliberately.

What the shaping engine does

Between code points and glyphs sits a shaper: HarfBuzz²⁶ on most Linux and web stacks, DirectWrite/Uniscribe on Windows, CoreText on Apple platforms. For Devanagari it runs a script-specific model that, roughly:

1. Splits the run into syllable clusters.
2. Identifies the base consonant of each cluster.
3. Applies OpenType²⁷ features from the font – nukt, akhn, rphf, blwf, half, pstf, vatu, cjct – to form half-forms, conjuncts and below-base forms.
4. Reorders pre-base matras such as ि so they sit to the left of the base glyph.
5. Applies positioning features and produces final glyph IDs with offsets.

Step 4 is the answer to the कि puzzle from the start of the post. The reordering is a rendering operation, performed by the shaper, using rules that depend on the font. It never touches the stored text.

This separation is also why Unicode does not encode every conjunct. Devanagari can form a very large number of consonant combinations. Encoding each visible shape separately would merge two different questions – what text is this and how should this text look – into one, and would make the encoding dependent on typographic fashion. Unicode answers the first question. Fonts and shapers answer the second.

Where this actually bites

The consequences are ordinary and frequent.

Truncation. Cutting a string at a fixed byte or code-unit offset can split a UTF-8 sequence, or split a cluster and leave a dangling virama or matra. Truncate on grapheme cluster boundaries, or at minimum on code point boundaries.

Cursor movement and deletion. Pressing backspace after typing क्ष should not leave क् behind. Editors need cluster-aware navigation. Note that for Indic scripts, the caret positions users expect are sometimes finer than grapheme cluster boundaries, which UAX #29 acknowledges as a legitimate tailoring.

Length validation. “Maximum 20 characters” needs a definition. Take the sentence भारत एक महान देश है। – that is 52 UTF-8 bytes, 20 UTF-16 code units, 20 code points, 16 grapheme clusters, and five words. Five plausible answers, and a varchar(20) column accepts or rejects it depending on which one the database counts. PostgreSQL varchar(n) counts code points, MySQL utf8mb4 counts characters, Oracle VARCHAR2 counts bytes or characters depending on how the column was declared.

Comparison and search. Without normalization, canonically equivalent strings fail to match. Without a collator, sort order is wrong for the nukta letters. Substring search on code point offsets can match across a cluster boundary.

Word segmentation. BreakIterator.getWordInstance does not treat the danda U+0964²⁸ as a word boundary, so the last token of a Hindi sentence comes back as है। with the punctuation attached. Sentence-ending punctuation that is not a full stop needs handling you would not write for English.

Reversal. Reversing a string by code point turns नमस्ते into a sequence that no longer renders as valid text. Reverse by grapheme cluster or not at all.

Regular expressions. . matches a code point in most engines, not a cluster. Character classes written for Latin text do not carry over.

The point

When we look at क्ष our eyes report one thing. The computer is handling three code points and nine bytes, and reports either one cluster or two depending on which segmentation library got linked. When we look at कि we see the vowel first, but it is stored second.

Neither representation is wrong. They are different layers of the same writing system. Unicode encodes the logical text, an encoding form turns it into bytes, a shaper interprets it, a font supplies the glyphs, and the renderer produces pixels. What appears on the screen is the end of that pipeline, not a picture of what is in memory.

A semantic network is useful because it forces you to say what the world contains and how one state can become another.

That sounds simple, but it is the hard part of many AI problems. Before an agent can search, reason, or explain a path, it needs a representation of the situation. The representation does not need to contain everything. In fact, a good representation usually leaves out most things. It needs to contain the distinctions that matter for the task.

Once those distinctions are explicit, the problem starts to look like a graph.

Nodes are states. Edges are transformations. A path through the network becomes a candidate solution. The agent’s job is no longer to “think” in some vague sense. Its job is to move through the represented space in a disciplined way.

Generate and test

The generate-and-test pattern is one of the simplest ways to use that structure.

The generator proposes a possible next state. The tester checks whether that state is legal, useful, or final. If the state does not work, the system generates another candidate. If it does work, the system keeps it and continues.

That pattern is easy to underestimate because it sounds brute-force. But the important design question is not only how candidates are generated. It is what the system remembers.

If the agent does not remember prior states, it can loop. It can regenerate the same state again and again, wasting time and appearing less intelligent than it really is. In a graph, memory is not a luxury. It is part of the search machinery.

So the generator has at least two responsibilities:

• Produce plausible next states.
• Avoid returning to states the system has already explored.

The tester also has at least two responsibilities:

• Reject invalid transitions.
• Recognize when the current state satisfies the goal.

Once those responsibilities are separated, the AI problem becomes easier to reason about. You can improve the generator without changing the tester. You can make the tester stricter without rewriting the whole search.

Why representation matters

The value of a semantic network is not the diagram itself. The value is the discipline it imposes.

It asks:

• What counts as a state?
• What relations connect states?
• Which transformations are legal?
• Which state is initial?
• Which state is final?
• What must be remembered to avoid cycling?

These questions show up everywhere in software, not just in AI courses. Workflow engines, build systems, dependency graphs, compilers, routers, game engines, and planning tools all depend on some version of this idea. A system behaves better when its state space is made explicit.

The practical lesson

Many AI discussions jump too quickly to model capability. But for knowledge-based AI, the representation often matters more than the algorithm.

A weak representation makes even a clever search look confused. A strong representation can make a simple generate-and-test loop surprisingly effective.

So a lot of the work in a knowledge-based system happens before the search runs at all. It happens when you decide what the agent is allowed to see.

My rules for naming have settled into a small set over the years. The core is Kent Beck’s four rules of simple design: passes the tests, reveals intention, no duplication, fewest elements. I first learned these rules from my mentor, Jacques Morel. Beck stated them for a design as a whole. I find they apply, almost without translation, to naming a variable in code or a field in an API schema. The rest comes from Strunk’s The Elements of Style and a short piece by Russ Cox. Here is how I read each rule when the thing being designed is a name.

Runs the tests

A declaration is a claim about what a variable will hold, and the type is the only part of that claim anything ever checks. The type should be able to test the value the variable must hold. Not everything is a String.

String email;   // nothing checks that this is an email
Email email;    // construction fails unless the value parses

A bare String passes every possible value, which means it tests nothing. An Email type that validates in its constructor turns every assignment into a test, and that test also runs in production, on every value the field receives. The same holds in an API definition: type: string promises nothing, while format, enum, pattern, and range constraints are the schema’s way of testing values before they reach your code. I wrote earlier about where types end and tests begin; this is the naming-side consequence. Choosing a type is part of choosing a name, because both are claims about the value.

Reveals intention

The variable in code and the field in an API or schema must say what it is for. If a reader has to look at usages to work out what a field means, the name has failed. This matters even more in an API than in code, because the consumer of a schema cannot read your implementation. They have the name, the type, and whatever description you wrote, and most, however, will read no further than the name.

No duplication

A name should not repeat what its surroundings already say. order.orderId says order twice; inside an Order, the field is just id. Suffixes like Value, Flag, String, and Data repeat the type, and the type is already visible in the declaration. I made the same argument about Request and Response suffixes on API models in Vibe Coding and the Baby Genius Problem. Say everything once, and only once – that holds for the words inside a name as much as for the code around it.

Fewest elements

Define only what you need. Once a field is in a schema, consumers start depending on it, and removing it becomes a breaking change. A field added because someone might need it later still has to be named, documented, validated, and migrated, without any current use to justify that work. If nothing reads it today, leave it out.

Omit needless words

Strunk’s rule 13 says a sentence should contain no unnecessary words. Applied to a name: customerEmailAddressString contains one useful word. String repeats the type, Address repeats what email already implies, and what remains after striking them is customerEmail – or, inside a Customer, just email.

Length follows scope

Russ Cox puts it as: a name’s length should not exceed its information content. A loop index that lives for three lines is i, and making it loopIndexCounter adds letters without adding information. A public field, read far from its declaration by people who never see the surrounding code, has to carry more.

These rules usually agree. When they conflict, I pick whichever name is easier for the reader. In a follow-up post I will take a badly named API model and apply these rules to it one pass at a time.

Parting Thoughts

I will just end with a thought from Donald Knuth. In his article introducing literate programming, he said:

…The practitioner of literate programming can be regarded as an essayist, whose main concern is with exposition and excellence of style. Such an author, with thesaurus in hand, chooses the names of variables carefully and explains what each variable means…

And this has been my endeavor.

Finding out how the package was installed

Removal starts with knowing what installed the file you can see:

$ which ghostty
/usr/bin/ghostty
$ dpkg -S /usr/bin/ghostty
ghostty: /usr/bin/ghostty
$ apt list --installed 2>/dev/null | grep ghostty
ghostty/now 1.3.1-0~ppa2 amd64 [installed,local]

dpkg -S maps a file back to the package that owns it, so this is a real apt/dpkg package, not a manually copied binary or a snap.¹ The [installed,local] marker says the package is installed but no configured repository offers it anymore — the PPA it came from is no longer in /etc/apt/sources.list.d/, so apt considers it a local package with no update source.

Before removing, the package shows in dpkg’s database like this:

$ dpkg -l ghostty | tail -1
ii  ghostty        1.3.1-0~ppa2 amd64        Fast, feature-rich, and cross-platform terminal emulator.

The two letters at the start are the package state. The first letter is the desired state — what you asked for — and the second is the current state. ii means desired “install”, currently “installed”: the normal state of an installed package.²

Remove

$ sudo apt remove ghostty

apt remove deletes the package’s files but keeps its conffiles — configuration files, typically under /etc, that dpkg tracks so your edits survive upgrades and reinstalls.³ Keeping them on removal is deliberate: if you reinstall the package later, your configuration is still there.

Confirm

Two checks. First, the binary is gone:

$ which ghostty
$

Second, the state in dpkg’s database changed:

$ dpkg -l ghostty | tail -1
rc  ghostty        1.3.1-0~ppa2 amd64        Fast, feature-rich, and cross-platform terminal emulator.

rc reads as desired “remove”, currently “config-files”: the package is removed, but its conffiles are still on disk and dpkg still has a record of it.² This is why a removed package keeps appearing in dpkg -l output — the entry is not a leftover installation, just the retained configuration. dpkg -L ghostty lists exactly which files remain.

Purge

To drop the conffiles and the database record too:

$ sudo apt purge ghostty

Purge works both on installed packages (it removes and purges in one step) and on packages already in the rc state. Afterward the package is gone from the database entirely:

$ dpkg -l ghostty
dpkg-query: no packages found matching ghostty

That is the end state: no files, no conffiles, no record.

If the package pulled in dependencies nothing else uses, sudo apt autoremove clears those as well; apt prints the candidates at the end of the remove step.

What the package manager will not clean

Everything above covers only files the package installed. Debian packages do not write to home directories, so anything the application or you created there stays after a purge. For Ghostty on this machine that was:

• ~/.config/ghostty/ — the application’s own configuration
• ~/.local/share/applications/com.mitchellh.ghostty.desktop — a user-level desktop entry I had added to override the launch command
• an alias ghostty=... line in ~/.bashrc and ~/.zshrc

The generic checklist after purging a package: its directory under ~/.config/ and ~/.local/share/, caches under ~/.cache/, aliases or environment variables in shell rc files, and — if you added one for the package — the repository entry under /etc/apt/sources.list.d/.

A final check that nothing is left:

$ which ghostty
$ dpkg -l ghostty
dpkg-query: no packages found matching ghostty
$ grep -ri ghostty ~/.bashrc ~/.zshrc ~/.config ~/.local/share/applications
$

References

Why I was trying ghostel in the first place

I run coding agents like Claude Code and OpenCode from the terminal, and I live in Emacs, so the obvious move is to run them in eshell. That does not work.

Eshell is not a terminal emulator. It is a shell written in Emacs Lisp that runs a command and inserts its output into a buffer. Modern coding agents are full-screen TUI applications — Claude Code, for instance, is built on Ink, a React renderer for the terminal.¹ Programs like that assume a real terminal: a PTY, raw-mode keyboard input, ANSI cursor addressing, the alternate screen buffer, and a constant stream of escape sequences repainting the screen many times a second.²

Eshell provides almost none of that. It interprets basic color codes through ansi-color and little else; it advertises TERM=dumb; its input model is line-oriented, so the agent never sees individual keystrokes; and there is no cursor positioning or alternate screen. So the agent’s UI arrives as garbled escape sequences, redraws pile up in the buffer instead of replacing each other, and interactive prompts do not respond to keys. Eshell’s escape hatch for this — listing the program in eshell-visual-commands so it runs under term.el — helps, but term.el is slow and its emulation is incomplete enough that the agents still misrender.³

Ghostel fixes this the same way vterm does: by embedding a real VT engine. A native module (written in Zig) links libghostty-vt and handles terminal state, rendering, and PTY I/O, while Elisp handles buffers and keymaps — the same two-layer design as emacs-libvterm, but with Ghostty’s engine instead of libvterm. That brings the Kitty keyboard protocol, synchronized output (DEC 2026), and OSC 8 hyperlinks, none of which libvterm supports.⁴ Claude Code and OpenCode render and respond correctly inside it.

Ghostel only needs libghostty-vt, not the Ghostty application. But using it made me curious about the terminal itself, so I installed Ghostty on my ThinkPad T430 — the same 2012 machine from my WezTerm post.

It would not open

Launching Ghostty from the desktop, the window flashed and closed. Running ghostty from a terminal showed why:

info: ghostty version=1.3.1
info(gtk): GTK version build=4.14.5 runtime=4.14.5
info(opengl): loaded OpenGL 4.2
warning(opengl): OpenGL version is too old. Ghostty requires OpenGL 4.3
warning(gtk_ghostty_surface): failed to initialize surface err=error.OpenGLOutdated
warning(gtk_ghostty_surface): surface failed to initialize err=error.SurfaceError

The system provides OpenGL 4.2; Ghostty’s renderer requires 4.3. The desktop launch fails the same way — there is just no terminal attached to show the log, so the window appears and dies.

I checked which GPU this machine has. glxinfo was not installed, but lspci was enough:

$ lspci -nn | grep -Ei 'vga|3d'
00:02.0 VGA compatible controller [0300]: Intel Corporation 3rd Gen Core processor Graphics Controller [8086:0166] (rev 09)

That is the Intel HD 4000, the integrated GPU on Ivy Bridge, the third generation of Core processors, released in 2012.⁵

Mesa’s drivers

On Linux, OpenGL is implemented by Mesa: one userspace project containing many hardware drivers, with the right one selected at runtime for the GPU present. This machine runs Mesa 25.2.8 on Ubuntu 24.04.

Intel GPUs are split across three Mesa drivers by hardware generation: i915 covers gen2–3, crocus covers gen4–7 (it replaced the older i965 driver), and iris covers gen8 (Broadwell) and newer. Ivy Bridge is gen7, so it is served by crocus — crocus_dri.so sits in /usr/lib/x86_64-linux-gnu/dri/ on this machine.⁶

Crocus exposes at most OpenGL 4.2 on Ivy Bridge, and GLSL 4.20 to match. GLSL is the language shaders are written in — shaders being the small programs a renderer runs on the GPU — and its version tracks OpenGL’s. OpenGL 4.3 is an umbrella over a bundle of features — compute shaders (GPU programs for general computation rather than drawing), shader storage buffer objects (blocks of GPU memory those programs can read and write), and others — and this generation never got the complete set.⁷ Haswell, one generation later, gets OpenGL 4.6. The ceiling is per GPU generation inside the driver, so no Mesa upgrade will move it.

Mesa also ships software rasterizers — renderers that do all of the GPU’s drawing work on the CPU instead. The main one is llvmpipe, which uses LLVM to generate fast machine code at runtime and implements OpenGL 4.5.⁸ That becomes relevant below.

Advertising a version the hardware does not have

The version an application sees comes from Mesa: glGetString(GL_VERSION) and glGetIntegerv(GL_MAJOR_VERSION, ...) on the context it created. Mesa has an environment variable, MESA_GL_VERSION_OVERRIDE, that makes it report a different version than the driver’s real one — 4.3FC means version 4.3, forward-compatible core profile, the variant of OpenGL with the old deprecated functions removed. It enables no functionality; it only changes the advertised number. The variable exists because hardware sometimes supports every extension an application actually uses while the driver stops short of exposing the version umbrella, and an application that only gates on the number will then run fine.⁹

$ MESA_GL_VERSION_OVERRIDE=4.3FC ghostty
info(opengl): loaded OpenGL 4.3
error(opengl): shader compilation failure id=2 message=0:1(10): error:
  GLSL 4.30 is not supported. Supported versions are: 1.10, 1.20, 1.30,
  1.40, 1.50, 3.30, 4.00, 4.10, 4.20, 1.00 ES, and 3.00 ES
warning(gtk_ghostty_surface): failed to initialize surface err=error.CompileFailed

In order: Ghostty’s version check now passes (“loaded OpenGL 4.3”). It proceeds to compile its shaders, which begin with #version 430. Mesa’s GLSL compiler, which still truthfully supports only up to 4.20 on this driver, rejects the directive. Surface initialization fails with CompileFailed instead of OpenGLOutdated. GTK’s own GSK renderer printed the same GLSL errors once the context claimed 4.3.

There is a companion variable, MESA_GLSL_VERSION_OVERRIDE, that would make the compiler accept a #version 430 directive too — but it also adds no functionality, so it would only move the failure into whichever 4.30 features the shaders actually use. The override moved the failure from the version check into the compiler, which settles the question: the limit is in the driver and hardware, not in Ghostty being overly strict.

Software rendering works

Mesa’s llvmpipe implements OpenGL 4.5, comfortably above Ghostty’s requirement:

$ LIBGL_ALWAYS_SOFTWARE=1 ghostty
info(opengl): loaded OpenGL 4.5

The surface initializes, the window opens, and Ghostty runs normally. Making this permanent takes two small changes: a user-level desktop entry at ~/.local/share/applications/com.mitchellh.ghostty.desktop that overrides the system one with

Exec=env LIBGL_ALWAYS_SOFTWARE=1 /usr/bin/ghostty --gtk-single-instance=true

and a shell alias for terminal launches:

alias ghostty="LIBGL_ALWAYS_SOFTWARE=1 ghostty"

Both work. But every frame is now rendered by the CPU.

Removing it

Ghostty’s selling points are its GPU renderer and the platform-native UI. On this machine the first one is gone — llvmpipe on an i5-3320M is functional but it is the opposite of what the terminal is designed around. I already use WezTerm, which runs hardware-accelerated on this same GPU (its OpenGL front end targets a version Ivy Bridge can provide), and I documented that setup earlier today. Running Ghostty in software-rendering mode gives me nothing WezTerm does not already do better here, so I uninstalled it and removed the desktop override and the alias.

Ghostel is unaffected by this. It links libghostty-vt directly and renders through Emacs, so it does not depend on the GPU. The part of Ghostty I actually needed — its terminal emulation — runs fine on this laptop, and I will keep using it through ghostel.

References

WezTerm is an open-source, cross-platform terminal emulator written in Rust by Wez Furlong. It is GPU-accelerated, configured in Lua rather than a static config format, and comes with a built-in multiplexer — panes, tabs, and workspaces — so it can take over the job tmux usually does. The same config file works on Linux, macOS, and Windows.

I first switched to WezTerm on my old ThinkPad T430, a 2012 laptop with an i5-3320M and 8 GB of RAM. Two things pushed me off GNOME Terminal on that machine: WezTerm felt a little faster, and copying and pasting took fewer steps. Before writing this post I went back to the same T430 and measured both claims, because an impression like “felt faster” can easily be placebo.

Was it actually faster?

I ran each test inside a real window of each terminal on the T430 (Ubuntu, GNOME on Wayland; WezTerm 20240203, GNOME Terminal 3.52 with VTE 0.76), six runs each, first run discarded as warm-up. The results fall into two groups.

So, the only thing that mattered to me was Row 1. WezTerm puts a usable window on screen in under a tenth of a second; GNOME Terminal takes about three quarters of a second even with its background server process already running. I open a terminal dozens of times a day, so this is the delay I notice most.

The second and third rows go the other way, and by a wide margin. When a command floods the screen with output, VTE (the library under GNOME Terminal) skips rendering frames it can’t keep up with, while WezTerm renders more of them. I checked whether WezTerm’s default WebGPU renderer was falling back to software on the T430’s ancient Intel HD 4000 by rerunning with front_end = "OpenGL" — the numbers were the same, so this is just how WezTerm behaves. It also uses about four times the memory.

So “a little faster” was not placebo, but it needs qualifying: WezTerm is much faster at opening a terminal, which I do constantly, and slower at dumping bulk output, which I do rarely. If you spend your day cat-ing huge logs on old hardware, GNOME Terminal is the faster terminal for you. For how I use a terminal, the trade-off favors WezTerm.

The copy-paste difference

You can check this one against the default keybindings (wezterm show-keys). In WezTerm, releasing a mouse selection runs CompleteSelection(ClipboardAndPrimarySelection), which puts the selected text in the system clipboard immediately; there is no separate copy step. Plain Ctrl+C and Ctrl+V also work: Ctrl+C copies when a selection exists and sends the interrupt signal when one doesn’t.

In GNOME Terminal, a mouse selection only goes to the X primary selection. Getting it into the clipboard needs an explicit Ctrl+Shift+C, and pasting into the terminal needs Ctrl+Shift+V. That extra step, repeated dozens of times a day, was the difference I had been feeling.

WezTerm adds two keyboard-only tools GNOME Terminal has no equivalent for: Quick Select (Ctrl+Shift+Space) overlays short labels on URLs, paths, and hashes visible on screen so you can grab one without touching the mouse, and Copy Mode (Ctrl+Shift+X) gives modal, keyboard-driven selection. URLs in output are clickable with a plain left click.

The rest of this post is the setup I ended up with: a Lua config with Emacs-style keybindings, workspaces, and per-machine overrides.

Installation

On Ubuntu and Debian, WezTerm has an apt repository:

curl -fsSL https://apt.fury.io/wez/gpg.key | \
  sudo gpg --yes --dearmor -o /etc/apt/keyrings/wezterm-fury.gpg
echo 'deb [signed-by=/etc/apt/keyrings/wezterm-fury.gpg] https://apt.fury.io/wez/ * *' | \
  sudo tee /etc/apt/sources.list.d/wezterm.list
sudo apt update && sudo apt install wezterm

There is also a Flatpak, and packages for most other distributions are listed at wezfurlong.org/wezterm. On macOS it’s brew install --cask wezterm; if you install to ~/Applications instead of /Applications, add $HOME/Applications/WezTerm.app/Contents/MacOS to your PATH so the wezterm CLI works.

The config file

WezTerm reads ~/.wezterm.lua (or ~/.config/wezterm/wezterm.lua). The config is a Lua program, which means conditionals, loops, and per-machine logic live in the same file as the settings. Surprisingly, my son is interested in Lua as well as it is used in Roblox. He is currently learning how to code in Lua.

The config builder gives better error messages when a key is misspelled:

local wezterm = require 'wezterm'
local config = wezterm.config_builder()

-- ... your configuration ...

return config

Fonts and ligatures

Pick a Nerd Font so prompt icons (Starship, Powerlevel10k) render correctly:

config.font = wezterm.font('JetBrainsMono Nerd Font', { weight = 'Medium' })
config.font_size = 14.0

-- Enable ligatures: => becomes →, != becomes ≠
config.harfbuzz_features = { 'calt=1', 'clig=1', 'liga=1' }

Other good Nerd Font options: FiraCode Nerd Font, Hack Nerd Font, CaskaydiaCove Nerd Font.

Color scheme

WezTerm ships with hundreds of built-in schemes, so there is no need to paste hex colors:

config.color_scheme = 'Catppuccin Mocha'

Run wezterm ls-colors to list them, or browse the color scheme gallery.

Window appearance

config.window_decorations = 'TITLE | RESIZE'
config.window_background_opacity = 0.92
config.window_padding = {
  left = 12, right = 12, top = 12, bottom = 12,
}
-- macOS only; ignored elsewhere
config.macos_window_background_blur = 20

On a machine as old as the T430 I keep the opacity at 1.0 — transparency costs compositing work for no benefit on a small screen.

Tab bar

config.use_fancy_tab_bar = true
config.hide_tab_bar_if_only_one_tab = false
config.tab_max_width = 30

Custom tab titles

Show the current directory name instead of the default process title:

wezterm.on('format-tab-title', function(tab)
  local pane = tab.active_pane
  local cwd = pane.current_working_dir
  local title = pane.title
  if cwd then
    local path = cwd.file_path or ''
    local folder = path:match('([^/]+)/?$') or title
    title = folder
  end
  if tab.is_active then
    return ' ● ' .. title .. ' '
  end
  return ' ' .. title .. ' '
end)

Status bar

Show the active workspace name and time in the right status area:

wezterm.on('update-right-status', function(window)
  local workspace = window:active_workspace()
  local time = wezterm.strftime('%H:%M')
  window:set_right_status(wezterm.format({
    { Text = '  ' .. workspace .. '  │  ' .. time .. '  ' },
  }))
end)

Keybindings: leader key, Emacs-style

Instead of memorizing modifier combos, I use a leader key, like tmux’s Ctrl+b or Emacs’s C-x. Press the leader, then a single key.

local act = wezterm.action
config.leader = { key = 'Space', mods = 'CTRL', timeout_milliseconds = 1000 }

After pressing Ctrl+Space, you have one second to press the next key.

Pane management

Modeled after the Emacs window commands (C-x 2, C-x 3, C-x 0, C-x 1), so the muscle memory transfers:

config.keys = {
  { key = '3', mods = 'LEADER', action = act.SplitHorizontal { domain = 'CurrentPaneDomain' } },
  { key = '2', mods = 'LEADER', action = act.SplitVertical { domain = 'CurrentPaneDomain' } },
  { key = 'o', mods = 'LEADER', action = act.ActivatePaneDirection 'Next' },
  { key = '0', mods = 'LEADER', action = act.CloseCurrentPane { confirm = true } },
  { key = '1', mods = 'LEADER', action = act.TogglePaneZoomState },
  { key = 'LeftArrow', mods = 'LEADER', action = act.ActivatePaneDirection 'Left' },
  { key = 'RightArrow', mods = 'LEADER', action = act.ActivatePaneDirection 'Right' },
  { key = 'UpArrow', mods = 'LEADER', action = act.ActivatePaneDirection 'Up' },
  { key = 'DownArrow', mods = 'LEADER', action = act.ActivatePaneDirection 'Down' },
}

Tab management

{ key = 'c', mods = 'LEADER', action = act.SpawnTab 'CurrentPaneDomain' },
{ key = 'b', mods = 'LEADER', action = act.ActivateTabRelative(-1) },
{ key = 'f', mods = 'LEADER', action = act.ActivateTabRelative(1) },
{ key = 'k', mods = 'LEADER', action = act.CloseCurrentTab { confirm = true } },

Resize mode

A modal resize mode where the arrow keys resize panes; Escape or Enter exits:

{ key = 'r', mods = 'LEADER', action = act.ActivateKeyTable {
  name = 'resize_pane', one_shot = false
}},

-- Define the key table
config.key_tables = {
  resize_pane = {
    { key = 'LeftArrow', action = act.AdjustPaneSize { 'Left', 2 } },
    { key = 'RightArrow', action = act.AdjustPaneSize { 'Right', 2 } },
    { key = 'UpArrow', action = act.AdjustPaneSize { 'Up', 2 } },
    { key = 'DownArrow', action = act.AdjustPaneSize { 'Down', 2 } },
    { key = 'Escape', action = 'PopKeyTable' },
    { key = 'Enter', action = 'PopKeyTable' },
  },
}

Utility shortcuts

(On macOS the palette is Cmd+Shift+P.) The copy mode and quick select defaults (Ctrl+Shift+X, Ctrl+Shift+Space) keep working; the leader bindings are just easier for me to remember.

Workspaces: project-based context switching

Workspaces are the feature that let WezTerm replace tmux for me. Each workspace is an isolated set of tabs and panes with its own working directory, and switching between them swaps the whole window contents.

Basic workspace commands

{ key = 'w', mods = 'LEADER', action = act.PromptInputLine {
  description = 'Enter new workspace name:',
  action = wezterm.action_callback(function(window, pane, line)
    if line then
      window:perform_action(act.SwitchToWorkspace { name = line }, pane)
    end
  end),
}},
{ key = 's', mods = 'LEADER', action = act.ShowLauncherArgs { flags = 'FUZZY|WORKSPACES' } },
{ key = 'd', mods = 'LEADER', action = act.SwitchToWorkspace {
  name = 'default',
  spawn = { cwd = wezterm.home_dir },
}},

Per-machine workspace shortcuts

Hard-coding project paths in a tracked config file is impractical — paths differ between machines. The fix is a local override file. Create ~/.wezterm_local.lua (not version-controlled):

return {
  workspaces = {
    { key = 'e', name = 'eventing', cwd = '/home/me/Work/eventing-platform' },
    { key = 'p', name = 'personal', cwd = '/home/me/Workspaces/Personal' },
    { key = 'a', name = 'api', cwd = '/home/me/Work/api-gateway' },
  }
}

Then load it dynamically from the main config:

local local_config = {}
local ok, loaded = pcall(dofile, wezterm.home_dir .. '/.wezterm_local.lua')
if ok and loaded then
  local_config = loaded
end

local workspace_defs = local_config.workspaces or {}

-- Dynamically register workspace keybindings
for _, ws in ipairs(workspace_defs) do
  table.insert(config.keys, {
    key = ws.key,
    mods = 'LEADER',
    action = act.SwitchToWorkspace {
      name = ws.name,
      spawn = { cwd = ws.cwd },
    },
  })
end

Now Leader e jumps to the eventing workspace, Leader p to personal, and so on, with each machine defining its own list.

Hyperlink rules: clickable Jira tickets and GitHub issues

WezTerm can detect patterns in terminal output and make them clickable:

config.hyperlink_rules = wezterm.default_hyperlink_rules()

-- Jira tickets: PROJECT-1234 → opens in Atlassian
table.insert(config.hyperlink_rules, {
  regex = [[\b(MYPROJECT-\d+)\b]],
  format = 'https://myorg.atlassian.net/browse/$1',
})

-- GitHub shorthand: owner/repo#123 → opens the issue/PR
table.insert(config.hyperlink_rules, {
  regex = [[\b([A-Za-z0-9_-]+/[A-Za-z0-9_.-]+)#(\d+)\b]],
  format = 'https://github.com/$1/issues/$2',
})

A plain left click follows the link (on macOS it’s Cmd+Click).

Per-machine font size

Different monitors need different font sizes. Use the hostname:

local hostname = wezterm.hostname()
local font_size = 14.0
if hostname:find('work') then
  font_size = 13.0
end
config.font_size = font_size

Or put it in ~/.wezterm_local.lua if you prefer:

-- ~/.wezterm_local.lua
return {
  font_size = 13.0,
  workspaces = { ... }
}

Managing the config in dotfiles

Keep wezterm.lua in your dotfiles repo and symlink it:

ln -sf ~/path/to/dotfiles/wezterm.lua ~/.wezterm.lua

The machine-specific ~/.wezterm_local.lua stays untracked. Add it to your dotfiles .gitignore if it lives in the same directory:

# Per-machine overrides (never tracked)
*.local.lua

This gives you a single tracked config that works on every machine, with local overrides for paths and preferences that differ.

Quick reference

Conclusion

WezTerm replaced tmux, a color scheme plugin, and a hyperlink plugin for me, all from a single Lua file, and it did so first on a machine with little performance to spare. The benchmarks put a number on the speed I had felt: it comes from window-open latency, not throughput, and GNOME Terminal still handles bulk output better. If you try WezTerm, start with the basics — font, theme, leader key — and add workspaces and hyperlink rules as you need them.

I will be using WezTerm and GNOME Terminal together for a while before deciding which one I will keep.

The paper surveys legal precedent around software, especially in medicine, and argues that the rise of AI weakens one of the assumptions that has historically protected software vendors from malpractice-style liability. That assumption is that human professionals remain the real decision makers, while software merely assists them.

For ordinary clinical decision-support systems, that line may seem plausible. A doctor reviews the output, applies professional judgment, and accepts or rejects the recommendation. But with narrow AI systems that develop deep expertise in a domain, the relationship becomes less clean. If a system produces a recommendation that a human reviewer cannot meaningfully audit, then “human oversight” can become more procedural than real.

That is the paper’s strongest point. AI makes it harder to rely on the fiction that a human independently reviewed the basis for a software recommendation.

Where I agree

I agree with the paper’s central concern. Technology companies and software professionals should not be immune from responsibility simply because the harm is mediated through code.

Software now helps decide medical treatment, credit access, hiring, public benefits, fraud detection, insurance pricing, and many other areas where mistakes can alter a person’s life. When systems operate at that level of consequence, the industry needs a more serious account of duty, care, review, and accountability.

The paper is also right to look at professional malpractice as a useful comparison. Older professions have already built ideas around competence, fiduciary responsibility, licensing, insurance, standards of care, and disciplinary consequences. Software does not need to copy that structure blindly, but it should learn from it.

Where the analogy weakens

The main weakness is that the doctor-software-engineer analogy does not map cleanly.

A doctor usually owns the professional act of diagnosis or treatment. Even when the doctor consults peers, the final professional decision is attributable to that doctor. Software is rarely produced that way.

A deployed AI product is usually the result of many decisions:

• Product defines the user problem and desired behavior.
• Data teams choose or prepare datasets.
• Engineers implement the system.
• Model teams tune behavior.
• Designers shape how users interpret the output.
• Legal and compliance teams approve risk language.
• Executives decide whether the product ships.

If harm occurs, it may be impossible or unfair to assign responsibility only to the individual engineer who wrote part of the implementation. The engineer may have seen the risk but lacked authority to block release. Or the risk may have emerged from a product decision, data limitation, or business constraint outside the engineer’s control.

That does not mean nobody is responsible. It means software responsibility has to be organizational, not merely individual.

What the paper leaves open

The paper is valuable as a survey and warning, but it does less to explain how responsibility should actually be assigned inside software organizations.

Several questions need more work:

• What would a software “standard of care” look like for AI systems?
• Which duties should belong to engineers, product leaders, data scientists, executives, and companies?
• Should licensing apply to all software engineers, or only to those working in high-risk domains?
• How should certifications, professional societies, or bodies such as IEEE shape enforceable practice?
• How do existing frameworks such as GDPR or human-rights declarations become operational inside product development?

The strongest version of this argument would move from analogy to mechanism. It would define what competent AI development requires, what evidence must be produced before deployment, and who is accountable when that evidence is ignored.

Bottom line

“Software Malpractice in the Age of AI” is worth reading because it names a real gap: AI systems are becoming professionally consequential without the professional accountability structure that older high-stakes domains developed over time.

But the solution cannot simply be “treat software engineers like doctors.” Software is too collaborative, too organizational, and too entangled with product and business incentives for that analogy to carry the whole burden.

AI malpractice, if the term is to mean anything useful, has to attach responsibility to the system of production. That includes the engineer, but it also includes the company that creates the incentives, approves the risks, and profits from the deployment.

Reference

Danny Tobey, “Software Malpractice in the Age of AI: A Guide for the Wary Tech Company”, AIES 2018.

Miner Town: Awakening is a dystopian science fiction novel that blends post-apocalyptic survival, coming-of-age, and rebellion against an oppressive system. The world is imaginative and immediately engaging. Miner Town feels harsh, believable, and internally consistent, and the contrast between the mining settlement and the pristine Trinity cities provides a compelling foundation for the story.

The strongest aspect of the novel is its cast. Gage, Camilla, Otto, and Ben each represent different strengths, and their relationships drive much of the narrative. Rather than relying solely on action, the story spends time developing why these characters become who they are.

My main problem is with the prose. There is a vivid metaphor or philosophical reflection in every paragraph. While individually well written, their cumulative effect occasionally slows the narrative and reduces the impact of the strongest moments. The book would hit harder if it were plainer most of the time. Related issue: several pieces of exposition get delivered twice by different characters. Cutting those would help the pace.

Themes of sacrifice and exploitation are handled well, mostly because the author trusts the setting to carry them instead of having someone announce them.

An ambitious debut. If you liked The Hunger Games, you will recognize the genre. I will definitely like to read the next one.

Rating: 4.5/5

Find the book on Amazon.

The first question is what the team still verifies by hand. Every manual verification habit is a candidate skill file. The content is nothing more than the steps a reviewer already performs before trusting a change: what to start, what to click, what to compare, which tests to run. The habit feels too obvious to document, which is exactly why it never gets documented. A useful test for the finished file is whether a new team member could follow it without asking anyone.

The next question is how the team decides that a piece of work is done. If the honest answer is that an experienced person looks at it and says so, the missing artifact is a measurable exit criterion. A worthwhile exercise is to take one recent change that was accepted on judgement and restate the acceptance in checkable terms: a response time under a stated number, a set of tests that must pass, a score from an audit tool. Not every kind of done converts into a measurement, but more of them convert than the first attempt suggests, and each conversion is one less decision that has to be relitigated per change.

Then there are the tasks people do at the same time every day. A morning routine of reading a channel, sweeping a queue, or checking a dashboard is a schedule that exists only in someone’s habits. Writing it down as an actual schedule is the small step. The more valuable step is to ask what change in the world the routine is standing in for, and whether the system where that change happens can announce it. A webhook or a queue alert replaces a guess about timing with a fact about the source.

The last question concerns streams: bug reports, support tickets, dependency alerts, anything that arrives continuously and gets triaged by a person. This is where a proactive routine could stand, and it is also where the diagnostic has to be most careful. The routine needs a scoped goal, a budget, a rate limit, a shutoff, and an owner. The owner is the deciding item. If no one can be named who is responsible when the routine acts wrongly, the routine is not ready to build, whatever the tooling makes possible.

The output of the exercise is a short list: the checks nobody has written down, the criteria that live in one person’s judgement, the schedules that exist as habits, and the streams with no owner for their automation. A team does not need to adopt all four loops, and most should not. It needs to know which artifact on that list to write first.

But the file underneath is not a grid. A text file is a one-dimensional sequence: a flat run of characters, h e l l o \n w o r l d, where the newline is a character that means “start drawing on the next row.” The buffer that holds the document in memory is the same, a sequence you index with a single number.

So every editor lives with a permanent translation problem. This post looks at how a few of them solve it: a from-scratch Rust editor I’m building (textr), the two it is modeled on (gedit and Emacs), and VS Code for a fourth point of comparison.

• The impedance mismatch
• Where it comes up in textr
• Why not just pick one and be done?
• How gedit does it: iterators over a tree, cursor-as-mark
• How Emacs does it: point is a number, everything else is derived
• How VS Code does it: a piece tree
• Side by side
• The choice follows the data structure and the dominant operation
• Notes
• References

The impedance mismatch

Two coordinate systems, describing the same text:

• Humans and screens think in 2D: (line, column).
• Files and buffers store 1D: a single offset into a sequence.

Figure 1 — The same caret, two coordinate systems. (line 1, column 1) on the left is flat index 4 on the right. The mapping between them is the editor’s job.

Bridging those two systems correctly and cheaply is one of the basic jobs of a text editor. Get it wrong and the cursor lands mid-character in a UTF-8 sequence, or an insert meant for one place writes three lines away.

Where it comes up in textr

This shows up in textr’s View, the cursor model.¹ The cursor is stored the way a user thinks about it:

struct View { line: usize, column: usize, goal_column: usize }

That is a 2D coordinate. But the buffer, a ropey rope, has a strictly 1D editing API:

doc.insert(char_idx, text);   // one index
doc.remove(char_idx..end);    // one index

insert doesn’t know what a “line” is. To insert a character where the cursor is, I have to convert (line, column) into a single flat index. That conversion is one function:

pub fn cursor_char_idx(&self, doc: &Document) -> usize {
    doc.line_to_char(self.line) + self.column
}

line_to_char(line) gives the flat index where that line starts; add column and you have the caret’s absolute position in the whole document. It’s tempting to think column already is the index — and on line 0 it is, because line 0 starts at 0. But look at "ab\ncd" with the caret on the d:

             index:  0   1   2   3   4
             char:   a   b   \n  c   d
                                     ^
   (line 1, column 1)  ->  line_to_char(1) + 1  =  3 + 1  =  4

column is 1, but the flat index is 4. column is “where I am on this line”; the flat index is “where I am in the file.” The rope only understands the second one. (The reverse exists too, char_to_line(idx), for when you have a flat position, like a search hit or a mouse click, and need to find the right row.)

Both conversions are cheap on a rope: ropey documents line_to_char and char_to_line as O(log n) each, and the + column is O(1), so moving a cursor between the two coordinate systems runs in O(log n) whichever way it goes.²

Why not just pick one and be done?

The harder question is not how to convert but which representation is the source of truth, and how it survives edits. There are a handful of strategies:

1. Store 2D, derive 1D on demand (textr’s choice). Keep (line, column) canonical; compute the flat index whenever the buffer needs it. Never store the index — a stored index goes stale the instant text changes ahead of it. Pro: vertical movement is native (Up/Down, and the “goal column” that remembers your column as you glide over short lines, are 2D ideas). Con: every edit must update the 2D cursor by hand, and 2D positions need clamping.
2. Store 1D, derive 2D on demand (Emacs’s choice). Keep a single integer offset canonical; compute line and column only to display or move by rows. Pro: dead simple — a position is one number. Con: line and column now cost a computation you mitigate with caches.
3. Store both, keep them in sync. Fast reads either way, but now two things must be updated on every edit or they drift.
4. Hand out position objects. An “iterator” that internally carries both representations. Cheap to read either way — but typically invalidated by edits, so you can’t hold one across a modification.
5. Persistent marks. The subtle problem with any raw offset: if I remember “position 487” and someone inserts 10 characters at the top of the file, 487 now points somewhere wrong. A mark is a position the buffer itself keeps updated as text moves around it.

The choice is not free-floating. It falls out of what the buffer is made of:

Figure 2 — The storage structure decides which coordinate is cheap, which in turn pushes the design toward 1D-canonical or 2D-canonical.³

Strategies 4 and 5 are not hypothetical. gedit uses both: iterators for cheap reads and marks for persistence. Emacs uses markers (strategy 5) alongside its 1D point.

How gedit does it: iterators over a tree, cursor-as-mark

gedit is built on GTK’s GtkTextView/GtkSourceView, backed by a GtkTextBuffer.⁴ Internally that buffer isn’t a flat array — it’s a tree (a B-tree) that indexes lines and character/byte offsets, so it can answer “where does line N start?” and “what line is offset K on?” in roughly O(log n). Both directions of our conversion are cheap by construction.

You never touch raw offsets directly. Instead you work with GtkTextIter — a small stack-allocated struct representing a position, obtained from either coordinate system:

GtkTextIter iter;
gtk_text_buffer_get_iter_at_line_offset(buffer, &iter, line, column); // from 2D
gtk_text_buffer_get_iter_at_offset(buffer, &iter, char_offset);       // from 1D

And crucially, an iter carries both representations at once — you can ask it for its line (gtk_text_iter_get_line), its column (gtk_text_iter_get_line_offset), or its absolute char offset (gtk_text_iter_get_offset). The 2D↔1D conversion is baked into the iterator, powered by that tree. GTK’s answer to “which is the source of truth?” is essentially “neither — the tree makes both cheap.”

The catch: a GtkTextIter is transient — invalidated the moment the buffer is modified. So you don’t store iters; you re-fetch them.

For positions that must persist across edits, GTK gives you GtkTextMark, a named position the buffer maintains automatically. gedit’s cursor is a mark named "insert" (the selection’s other end is "selection_bound"). To render the caret, gedit gets an iter at the "insert" mark and reads its line and column; to type a character, it inserts at that iter. The persistence problem (strategy 5) and the conversion problem (strategy 4) are handled by two separate abstractions.

Figure 3 — gedit with this post’s draft open. The status bar reads “Ln 5, Col 12”, the 2D face of the "insert" mark.

How Emacs does it: point is a number, everything else is derived

Emacs comes at it from the opposite end. Its buffer is a classic gap buffer — one big array of characters with a movable gap where edits happen, which makes insertion and deletion at the cursor very cheap.⁵

Position in Emacs is point: a single integer, the character offset of the caret (1-based — point-min is 1; positions sit between characters). Point is 1D and canonical. Almost every primitive takes or returns a buffer position as a plain integer.

Line and column are not stored — they’re computed when asked:

(line-number-at-pos)   ; scan / caches to count newlines before point
(current-column)       ; scan back to line start, honoring tab-width and char widths

current-column shows that “column” is subtler than a character count: Emacs makes a tab advance to the next tab stop and accounts for wide characters, so the visual column and the character column differ. (textr, for now, uses character columns, a documented simplification.)⁶

Computing line numbers means scanning for newlines — O(distance) — so Emacs keeps caches to amortize it (line-number lookup on a multi-megabyte buffer is a real, historically-tuned concern). For persistence, Emacs has markers — objects that hold a position and are automatically nudged as text is inserted or deleted before them, with an insertion type that decides whether a marker sticks or advances when text lands exactly on it. Same idea as GTK’s marks, different name.

Emacs also separates character positions from byte positions (position-bytes) in multibyte buffers, so ordinary code can index by character and never split a multibyte sequence, the same reason textr indexes by char, never by byte.

Figure 4 — Emacs with the same file open. The mode line reads (12,4), the line and column derived from point.

How VS Code does it: a piece tree

VS Code reaches a similar place to textr from a different direction.⁷ Its buffer is a piece tree: a piece table whose pieces hang off a balanced red-black tree, where each node caches the text length and line-break count of its subtree, so a lookup by line or by offset walks the tree in roughly O(log n) instead of scanning. Its canonical position is 2D, like textr’s — the editor keeps the cursor as a Position of lineNumber and column, both 1-based, and the flat offset is derived on demand through getOffsetAt and its inverse getPositionAt. The reason for keeping 2D canonical differs from textr’s, though: the whole editor API is written in terms of Position, so 2D is the natural currency for the interface, not because vertical movement is the hardest job.

One detail sets VS Code apart from the other three. It measures column and the offset in UTF-16 code units, not characters, so an emoji outside the Basic Multilingual Plane counts as two columns and a grapheme built from combining marks spans several units. textr, gedit, and Emacs all index by char (Unicode scalar values); VS Code indexes by UTF-16 unit, and the two do not agree on where a given column falls.

Side by side

The choice follows the data structure and the dominant operation

There is no universally correct answer, only trade-offs that fall out of two things: what the buffer is good at, and which operation you do most.

• Emacs’s gap buffer makes offsets cheap and edits-at-point cheap, so 1D-canonical is the simplest choice; it pays for line and column with scans and caches.
• gedit’s tree makes both directions cheap, so it can afford to hide the whole question behind iterators and lean on marks for persistence.
• textr’s rope also makes both directions cheap (O(log n) either way), so the choice was free. I picked 2D-canonical because the View’s busiest and hardest job is vertical movement with a goal column, which is a 2D idea. The representation matches the operation it does most.
• VS Code’s piece tree makes both directions cheap like gedit’s, and it too keeps a 2D Position canonical — but for a different reason than textr: its whole editor API speaks in positions, so 2D is a matter of interface rather than of any single dominant operation.

One point is easy to miss: a raw offset is only correct for one moment. Once the text changes ahead of it, an old index points to the wrong place. That is why converting on demand (textr) or letting the buffer maintain the position (marks and markers) is safer than storing the number and reusing it. textr avoids the problem today because it has a single cursor that updates itself; when it grows multiple cursors or collaborative editing, it will want marks too, for the same reason every editor eventually adopts them.

Notes

• GtkTextIter is a value, not a handle. It’s valid only until the next buffer mutation; treat it as a snapshot and re-fetch after any edit. Persistent positions are GtkTextMarks, not stored iters.
• Marker insertion type decides the tie-break: when text is inserted exactly at a marker, does the marker stay put or advance past the new text? Emacs lets you choose per-marker; it’s the kind of detail that quietly determines whether your saved position feels “before” or “after” an edit.
• Character vs byte positions are different numbers in any multibyte buffer. textr indexes by char throughout and never does byte math (ropey also exposes byte-index methods; textr uses only the char ones) — which is why inserting a precomposed 'é' (U+00E9: one char, two UTF-8 bytes) advances the cursor by exactly one column. A decomposed 'é' (e + a combining accent) is two chars and would advance by two — so this assumes NFC-normalized input; char indexing is not grapheme indexing.
• ropey’s phantom trailing line. A rope for "a\nb\n" reports three lines — "a\n", "b\n", and a final empty "". The caret may rest on that empty line but no further; textr’s line_len_chars helper strips the trailing \n so “end of line” lands before it, not after.
• “Column” is not “display column.” This post (and textr, today) uses character columns. A real editor’s visible column has to account for tab stops and wide/zero-width characters — which is exactly what Emacs’s current-column does and where grapheme-cluster segmentation (UAX #29) eventually comes in.
• textr is a learning project — a from-scratch gedit clone I’m building to learn Rust, with a headless, UI-agnostic core and thin frontends. The 2D↔1D bridge above is one small, load-bearing piece of its View.

References

1. How is code compiled?
2. How is it distributed?
3. Where is encapsulation enforced?

Java accumulated its answers over time. Source files and packages came from the language, JARs became the distribution unit, and JPMS arrived in 2017 to add strong encapsulation and explicit dependency readability.

Rust and Cargo started with a more integrated model, designed together around a standard package and build system.

The two systems answer the same three questions but put the answers in different places. Java spreads them across pieces assembled over two decades — packages and the module system in the language, JARs and Maven outside it — that must stay consistent without anything forcing them to. Rust concentrates compilation and encapsulation in the crate and distribution in the Cargo package, with far less overlap.

The terms

Both toolchains reuse a few of the same words for different things, so it helps to fix the vocabulary before the code.

Rust / Cargo

• Crate — the unit the compiler compiles, and the unit of encapsulation. A crate is either a library crate or a binary crate.
• Package — what Cargo versions, builds, and publishes, described by Cargo.toml. A package holds at most one library crate and any number of binary crates.
• Module (mod) — a namespace inside a crate. Modules form a tree, and that tree is where visibility is enforced.
• Workspace — a set of packages that share one Cargo.lock and one build.

Java / Maven / JPMS

• Package — a namespace declared per source file (package in.systemhalted.gateway.api;).
• Module (JPMS) — a named group of packages with a module-info.java that declares what it reads and what it exports. The unit of strong encapsulation.
• Artifact (Maven) — the versioned, publishable unit, identified by groupId:artifactId:version and shipped as a JAR.
• JAR — the packaging and distribution format.

Shared

• Dependency — code from another package, artifact, crate, or module that this component uses. Maven resolves and fetches a Java dependency, while JPMS separately determines module readability through requires. Cargo resolves a Rust dependency and makes the corresponding crate available to the compiler from the same manifest declaration.

The words already collide. A Rust crate is closest to a JPMS module, a Rust module to a Java package, and a Cargo package to a Maven artifact. None of these is exact — a crate resembles a JPMS module in encapsulation and dependency structure more than in the mechanics of compilation — and “module” means almost opposite-scale things in the two worlds. The rest of this post lines these up against the three questions and shows where the words look alike but aren’t.

The same project, twice

Consider a small API gateway library with a public API, a configuration type, and an internal routing engine.

Here is the Java version, built with Maven and JPMS:

gateway/
├── pom.xml
└── src/main/java/
    ├── module-info.java
    └── in/systemhalted/gateway/
        ├── api/
        │   └── RouteHandler.java
        ├── config/
        │   └── GatewayConfig.java
        └── internal/
            └── Router.java

And the Rust version:

gateway/
├── Cargo.toml
└── src/
    ├── lib.rs
    ├── api.rs
    ├── api/
    │   └── handlers.rs
    ├── config.rs
    ├── internal.rs
    └── internal/
        └── router.rs

Four files reveal most of the architectural difference: pom.xml, module-info.java, Cargo.toml, and lib.rs. They do not map one-to-one. Understanding why is most of understanding the two systems.

The four files

pom.xml: identity and dependencies

The POM belongs to Maven, not to the Java language. It declares the artifact’s coordinates and the dependencies required to build it:

<groupId>in.systemhalted</groupId>
<artifactId>gateway</artifactId>
<version>1.4.0</version>
<dependencies>
    <dependency>
        <groupId>com.fasterxml.jackson.core</groupId>
        <artifactId>jackson-databind</artifactId>
        <version>2.17.1</version>
    </dependency>
</dependencies>

Maven resolves the dependency, downloads the JAR, and constructs the classpath or module path. javac knows nothing about the POM.

module-info.java: readability and exports

The module descriptor lives at the root of the Java source tree, at src/main/java/module-info.java. It is part of the Java language and is compiled by javac.

module in.systemhalted.gateway {
    requires com.fasterxml.jackson.databind;
    exports in.systemhalted.gateway.api;
    exports in.systemhalted.gateway.config;
    // in.systemhalted.gateway.internal is not exported
}

The descriptor declares two boundaries: which modules this module reads, and which packages it exposes.

Jackson therefore appears twice. The POM tells Maven to obtain the JAR; the descriptor tells JPMS that this module may read it. These are separate systems, and nothing forces them to agree: a dependency can sit on the module path without a matching requires, or be declared with requires without being supplied by the build. The two names even differ — the Maven artifactId is jackson-databind, the JPMS module name com.fasterxml.jackson.databind. Coordinate and module identity are separate namespaces.

Cargo.toml: the package manifest

Cargo.toml describes the Cargo package:

[package]
name = "gateway"
version = "1.4.0"

[dependencies]
serde_json = "1"

The gateway package here is a single library crate, src/lib.rs. Add a src/main.rs and it would hold two crates, a library and a binary. The crate is where visibility and module boundaries apply.

Declaring a dependency in Cargo.toml both resolves it and makes it available during compilation; Cargo invokes rustc with the corresponding --extern arguments. There is no separate language-level file holding a second copy of the dependency declaration.

lib.rs: the crate root

lib.rs is not a manifest. It is source code — the root of the library crate and the starting point of the crate’s module tree:

// src/lib.rs
pub mod api;
pub mod config;
mod internal;

pub mod api; loads src/api.rs and makes the module publicly reachable; mod internal; loads src/internal.rs but does not make it reachable outside the crate.

This is where part of the Java module descriptor’s role moves into source. The descriptor’s exports in.systemhalted.gateway.api; and the crate root’s pub mod api; are not exact equivalents, but serve the same purpose: defining the externally reachable surface. In Rust the dependency side lives in Cargo.toml, the visibility side in ordinary source.

Where the module tree comes from

The deeper difference is not the file names. It is who defines the structure.

Java packages are declared independently

Every Java source file declares its package, such as package in.systemhalted.gateway.internal;. Build tools conventionally mirror that name in the source directory, as in/systemhalted/gateway/internal/Router.java. But the package declaration defines membership: no parent package registers the class, and no other source file has to mention it.

A Java package is a flat namespace. The dots suggest hierarchy, but there is none: a.b and a.b.c are separate packages, and a.b.c is not semantically nested inside a.b.

Rust modules form an explicit tree

In Rust, a file does not join the crate merely by existing on disk; a parent module must declare it. In src/lib.rs, mod internal; brings src/internal.rs into the crate. In turn, src/internal.rs’s pub mod router; brings src/internal/router.rs into the tree, and src/api.rs’s pub mod handlers; brings src/api/handlers.rs. Remove the mod router; declaration and the compiler never reads router.rs, even if the file remains on disk. The filesystem follows the module tree; it does not create it.

This explains why the layout carries both internal.rs and an internal/ directory: internal.rs defines the module, internal/ holds its child modules. A leaf module such as config needs only config.rs. The older internal/mod.rs layout still works, but the internal.rs-plus-internal/ style is now common.

Visibility follows the tree

Once the tree exists, visibility is evaluated along paths through it. Consider:

// src/internal/router.rs
pub struct Router { /* ... */ }

pub(crate) fn rebuild_routes() {
}

pub(super) fn debug_dump() {
}

pub(in crate::internal) fn merge() {
}

fn parse_segment() {
}

These declarations represent different visibility scopes.

pub is bounded by the path

This is the first rule that often surprises Java developers. pub struct Router; does not necessarily make Router reachable outside the crate; every module along the path must also be public. The full path is crate::internal::router::Router, but lib.rs declared mod internal;, not pub mod internal;. The private internal module blocks the path: Router is public within the visibility that path allows, but the path itself is not externally reachable.

Visibility belongs to both the item and its route through the module tree.

Visibility can name an ancestor

Rust can express scopes relative to the tree: pub(crate) is visible throughout the crate, pub(super) within the parent module and its descendants, pub(in crate::internal) within the named module and its descendants. The path in pub(in path) must name an ancestor of the current module. A function inside crate::internal::router cannot declare pub(in crate::api), because api is a sibling branch, not an ancestor.

Java has no equivalent, because its packages do not form a semantic tree. For top-level types and cross-package access, Java mainly offers public and package-private; class members also have private and protected, but none express visibility to a named package subtree.

Before JPMS, a package named internal was mostly a warning:

This is internal. Please do not use it.

JPMS made that boundary enforceable across modules by withholding exports. But inside the module, Java remains flat.

Visibility, keyword by keyword

External reachability is path-based in Rust and package-based in JPMS. Each Rust modifier grants a scope defined by the module tree; the nearest Java construct is often not a keyword at all, because Java expresses external reachability through the module descriptor rather than a modifier on the declaration.

Java approximates pub(crate) by placing a public class in a package omitted from exports — not a Java visibility level, but an effect of the module boundary. Rust privacy also differs from Java package privacy: a private Rust item is visible to its defining module and that module’s descendants, while a package-private Java member is visible to all code in the same package.

Imports reveal the architecture

The different namespace models also shape imports.

Java:

import java.util.List;
import in.systemhalted.gateway.api.RouteHandler;

Rust:

// src/internal/router.rs
use std::collections::HashMap;
use crate::api::RouteHandler;

And from a child module, use super::Router;.

A Java import aliases a fully qualified name. That name belongs to a global package namespace and says nothing about where the importing class sits relative to the imported one. There is no Java equivalent of super::, because Java packages have no parent-child relationship.

A Rust use navigates the module tree:

• crate:: starts at the crate root
• self:: starts at the current module
• super:: starts at the parent module
• an external crate is named directly

The module tree is therefore more than an encapsulation mechanism. It is the coordinate system for names throughout the crate.

Re-exports and transitive readability

Rust and JPMS both allow one component’s API to depend on another, but the mechanisms are very different.

JPMS propagates readability

module in.systemhalted.gateway.api {
    requires transitive in.systemhalted.gateway.types;
}

Any module that requires gateway.api also reads gateway.types, without declaring gateway.types itself. The dependency remains a separate module: its classes keep their original package and module identity, and consumers can access only the packages that gateway.types exports. requires transitive changes the readability graph.

Rust republishes names

In src/lib.rs, pub use gateway_types::RouteConfig; gives consumers a public path through the current crate, gateway::RouteConfig. Rust can also re-export an item from a private internal module with pub use crate::internal::router::Router;: the internal module remains private, but Router becomes available at gateway::Router.

This works because Router is public and reachable from lib.rs. A re-export cannot override privacy; it can only republish an item the re-exporting module can already access.

This gives Rust an important form of indirection: the implementation can live at crate::internal::router::Router while the public API stays gateway::Router, and the internal module tree can change without changing the path exposed to consumers.

Java cannot do this directly, because a class’s package is part of its identity.

Scaling up: modules and workspaces

Real projects rarely remain a single component. Suppose the gateway is split into two libraries, gateway-api and gateway-core.

Maven multi-module build

gateway/
├── pom.xml
├── gateway-api/
│   ├── pom.xml
│   └── src/main/java/
│       ├── module-info.java
│       └── in/systemhalted/gateway/api/
└── gateway-core/
    ├── pom.xml
    └── src/main/java/
        ├── module-info.java
        └── in/systemhalted/gateway/core/

The root POM aggregates the modules:

<packaging>pom</packaging>
<modules>
    <module>gateway-api</module>
    <module>gateway-core</module>
</modules>

gateway-core declares the Maven dependency in its POM and the JPMS dependency in its module descriptor.

Cargo workspace

gateway/
├── Cargo.toml
└── crates/
    ├── gateway-api/
    │   ├── Cargo.toml
    │   └── src/
    │       └── lib.rs
    └── gateway-core/
        ├── Cargo.toml
        └── src/
            └── lib.rs

The root manifest defines the workspace:

[workspace]
members = ["crates/*"]

gateway-core declares a path dependency:

[dependencies]
gateway-api = { path = "../gateway-api" }

The crates/ directory is only convention; the members declaration defines the workspace. Each member remains a normal Cargo package with its own manifest, source tree, and crate root. A workspace adds organization above packages without weakening the boundaries between their crates.

Dependency management

The Maven and Cargo mappings are close but not identical. A Cargo workspace shares one resolved dependency graph through Cargo.lock. Centralized dependency declarations belong in [workspace.dependencies], and member packages inherit them with serde = { workspace = true } — closer to Maven’s dependencyManagement than the lock file itself.

At the build level, both construct a dependency graph and build components in order: cargo build from the workspace root resembles a Maven reactor build like mvn package from the parent project.

The architectural consequence is the same: a crate boundary inside a workspace is still a real boundary. gateway-core sees only what gateway-api exposes publicly, just as one JPMS module sees only the packages another exports. Splitting a project into crates or JPMS modules is therefore not directory organization; it changes the enforceable architecture.

Split packages and legacy constraints

JPMS arrived after two decades of classpath-based Java. That history created migration mechanisms and compatibility layers:

• automatic modules
• the unnamed module
• classpath fallback
• command-line overrides
• split-package failures

A split package occurs when two or more named modules define types in the same Java package. JPMS rejects that arrangement: a given package must belong to a single module. Rust rules out the analogous structure by construction — a Rust module belongs to one crate’s module tree and is introduced by declarations within that crate. No module spans two crates. There is no pre-crate Rust ecosystem to accommodate.

The escape hatch survives, too. Drop a JAR onto the legacy --class-path instead of the module path and its packages fall back into the unnamed module, outside JPMS entirely — strong encapsulation holds only as long as everyone stays on the module path. Rust has no such fallback: nothing sits outside the crate system, so its boundaries always hold.

This is not evidence that Rust’s designers were smarter; they had a different starting point. But the consequence matters: crate boundaries apply uniformly across the ecosystem, and tools can assume the model is real.

The runtime model

JPMS also governs deep runtime reflection. Java frameworks often inspect constructors, fields, and methods reflectively; whether that access is permitted depends on ordinary access checks and whether the containing package is open. A module can open one package to a specific framework:

module in.systemhalted.gateway {
    opens in.systemhalted.gateway.config
        to com.fasterxml.jackson.databind;
}

The command line can also modify the boundary with --add-opens.

Rust has no general runtime reflection facility comparable to Java reflection. Its major metaprogramming mechanisms — procedural macros and derive macros — operate during compilation, expanding into ordinary Rust code that remains subject to the language’s visibility rules. Rust therefore needs no equivalent of JPMS opens; the problem does not arise, because the runtime model is different.

The mapping

What each language should envy

Return to the three questions. Rust answers all of them inside the language and its build tool, the same way for every crate. Java answers them across pieces assembled over twenty years. That difference is the source of what each side can envy.

Java developers should envy Rust’s uniformity. Every Rust crate, from the standard library to a small third-party library, participates in the same crate and module model. There is no parallel classpath world, no automatic-module transition state, no split-package compatibility problem, and no reflective framework asking for runtime access to private implementation details.

Rust developers should envy the explicit JPMS boundary document. A module-info.java file states, in one compact artifact:

• which modules are readable
• which packages are exported
• which packages are opened
• which services are used or provided

Cargo can expose the resolved dependency graph through cargo metadata, and Rust tooling can resolve public re-exports, but a crate’s external namespace is assembled through source declarations rather than summarized in one descriptor. For architectural governance at module and package granularity, JPMS provides a cleaner artifact.

Neither system describes the complete public API: an exported Java package still has to be inspected for its public classes and methods, and a Rust crate still has to be analyzed for its public items and re-exports. But JPMS makes the high-level boundary unusually explicit.

Rust could make the crate the language’s central architectural unit, and Cargo built its package and dependency model around it: one place answers compilation and encapsulation, one place answers distribution. JPMS had no such freedom. It had to coexist with packages, JARs, the classpath, reflection-heavy frameworks, and two decades of existing code; its answers to the three questions were already spread apart before it arrived.

That historical constraint explains most of the irregularity in JPMS. It also explains most of the uniformity in Cargo.

The turn-based loop comes first because its artifact is the foundation. Working turn by turn with an agent forces a team to answer a basic question: how do we know a change is good? Writing the answer down as a runnable check produces the skill file. Nothing about this requires autonomy. It only requires noticing what a person does before accepting a result, and recording it.

Second is the goal-based loop, which consumes that check. An exit criterion is only useful if there is a way to evaluate it, and the evaluation is the check the turn-based stage wrote down. Defining the criterion is work of its own. A team has to say what done means in language precise enough for an evaluator to apply, and that precision usually does not exist yet. Producing it is the real output of this stage.

After the criterion is in place, a schedule or an event trigger follows, and this is the rung where teams most often stall. A schedule is the easiest of the four artifacts to create, so it tends to appear early and stay. A prompt that runs every morning feels like automation, and nobody revisits it. The better path is to treat the schedule as provisional from the day it is created, and to replace it with an event trigger when the source system offers one. A calendar entry is a guess about when the world changes. An event is a report that it did.

A proactive routine can only come last, because it consumes everything: the check, the criterion, the trigger, and the operational guardrails around all three. At this stage the question stops being technical. A routine that touches production systems needs an owner, a budget, and a way to be turned off, and those are organisational decisions. A platform team can supply the machinery. Only the owning team can supply the ownership.

Skipping a rung defers work rather than removing it. A proactive routine adopted before the exit criterion exists will need that criterion defined during an incident instead of during design. A goal-based loop adopted before the check is written down will evaluate against an evaluator’s guess. The artifacts get produced either way. The order decides whether they are produced deliberately or under pressure.

So the most capable loop is not the right starting point. The right starting point is the artifact underneath the rung a team is already on, finished and written down before the next step up.

In a turn-based loop, the check lives in your judgement. The agent gathers context, does the work, checks itself, and hands back a result. You decide whether to accept it, redirect, or discard. Your judgement does not scale past a certain volume of work. What helps is writing the check down as something the agent can run before it comes back to you. Start the dev server, exercise the changed control, take screenshots before and after, run the affected tests. A skill file that encodes these steps moves the check from one person’s head to a file the whole team can read and improve. Every loop above this one assumes that file exists.

A goal-based loop moves the check into the exit criterion. You state what done looks like, and an evaluator model tests the condition each time the agent tries to stop. A measurable criterion, such as a Lighthouse score or a latency threshold on a specific endpoint, stops the loop on something you can defend after the fact. A vague criterion stops the loop early, because the agent convinces itself it is done, or lets it run long, because there is no way to be sure. Most teams do not have exit criteria for their own work written in language a person could act on, let alone an agent. The first attempt at a goal-based loop usually exposes that gap.

With a time-based loop, the trigger carries the check. The same prompt runs every N minutes against the current state of the world: summarising a channel, sweeping a queue, checking a pull request for review comments. What usually breaks here is the interval, not the prompt. A short interval against a slow-moving source wastes tokens, and a long interval against a fast-moving source misses changes that need a response. When the source system can emit an event, such as a webhook or a queue depth alert, an event trigger ties each run to a change that actually happened. A time-based loop works better as a stage on the way to an event-driven one than as a destination.

A proactive loop spreads the check across the whole system around the agent. A routine watches a stream of bug reports, alerts, or tickets, and runs a goal-based loop for each item until someone turns it off. This is a queue consumer whose judgement comes from a model, so it needs the same properties as any production consumer: idempotency, rate limits, spend caps, observability, a shutoff, and an owner. The Anthropic post covers token budgeting and routing routine subtasks to smaller models, which matter for cost. The harder question is ownership. A routine without a clear owner will eventually do the wrong thing with nobody responsible for noticing.

Three of these places build on each other directly. The check written down at the turn-based level is what the goal-based loop evaluates, and the criterion defined at the goal-based level is what the proactive routine applies to each item on the stream. The trigger sits beside this chain rather than inside it: a schedule or an event source decides when the runs happen, not whether their results can be trusted. A proactive routine built on unverified goals does not automate anything useful. It repeats whatever was already broken, at a cost that grows with how often it runs.

So the loops are the visible layer, and the durable artifacts sit underneath them: the skill file, the exit criterion, the trigger, and the observability around the routine. A team can locate its missing artifact by looking at what it still does by hand. If UI changes are verified manually at the end of every session, the skill file has not been written. Case-by-case judgement on whether a fix is done usually means there is no measurable exit criterion. A script that someone remembers to start every morning is a schedule that has not been written down, and after that an event trigger. Hand triage of a steady stream of reports is a routine that has not been built yet. These artifacts outlive the specific prompt, the specific model, and the specific team. Writing them down is the work the loops post points to.

That post was about the boundary between what a type system can prove and what only a test can. There is a separate limitation worth looking at, one that has little to do with that particular bug. An example test only runs the input I give it. My save test proves something about the string abcdef written to a temporary file, and nothing about the other inputs the function will see. I chose the input, so the input agrees with me. Writing more example tests does not remove this, because I choose those inputs too.

From examples to a property

Take insert and delete on the document model. The usual way to test them is with an example.

#[test]
fn insert_then_delete_restores_buffer() {
    let mut doc = Document::from_str("hello");
    doc.insert(5, " world");        // "hello world"
    doc.delete(5, 11);              // back to "hello"
    assert_eq!(doc.text(), "hello");
}

This test passes, and it is not a bad test. But look at what it actually asserts: that inserting " world" at index 5 of "hello", and then deleting that range, gives back "hello". That is one point in a large space of possible inputs. The rule I care about is not about "hello" at all but that for any document, inserting a string and then deleting exactly that range returns the original document. I wrote a test about one case of the rule but I can do better through property-based testing.

Property-based testing lets me state the rule and let the framework choose the inputs. Instead of picking the input, I describe the range of valid inputs, state what must be true, and the framework generates many cases and tries to find one that fails. In Rust the common tool for this is the proptest crate¹.

use proptest::prelude::*;

proptest! {
    #[test]
    fn insert_then_delete_is_identity(base in ".*", ins in ".*", at in 0usize..=usize::MAX) {
        let mut doc = Document::from_str(&base);
        let at = at % (doc.len() + 1);          // an offset into the buffer
        let before = doc.text().to_string();

        doc.insert(at, &ins);
        doc.delete(at, at + ins.len());

        prop_assert_eq!(doc.text(), before);
    }
}

proptest runs this body a few hundred times with different generated values, and the test passes only if the property holds for all of them. I am no longer asserting a fact about "hello". I am asserting a fact about insert and delete.

Reading the generator line

Most of the work in that test is in the signature, so it is worth going through it slowly.

Each name in strategy clause defines one generated argument. A strategy is the proptest term for something that knows two things: how to produce a random value, and how to shrink that value toward something smaller when a test fails. On each run, proptest draws a value from each strategy and calls the body with them. The in reads a little like the in of a for loop, but it is not walking a fixed list. It is drawing from a described set of inputs.

base in ".*" and ins in ".*" generate strings. The literal ".*" is read as a regular expression, and the strategy produces strings that match it. In that expression . means any character and * means zero or more of them, so ".*" matches almost any string: the empty string, "hello", a single emoji, a two-byte character such as "é", whitespace, or control characters. This wide range is deliberate. ".*" will produce characters that take more than one byte, and those are the inputs I do not think to type by hand, which turns out to matter here. If I wanted a narrower set of inputs I would use a narrower expression, such as "[a-z]{1,10}" for one to ten lowercase letters, or "[0-9]+" for a run of digits. Here I want the widest set I can get.

at in 0usize..=usize::MAX generates an integer. The strategy here is an ordinary Rust range. proptest provides a Strategy implementation for the standard range types, so a range can be handed to it directly as a generator that produces values inside the range. Reading the bounds from left to right: 0usize is the low end, zero, written as a usize, which is Rust’s pointer-sized unsigned integer and the type used for indexing. The ..= operator makes the range inclusive, so the upper value is part of the range. usize::MAX is the largest value a usize can hold. The usize suffix on 0usize is not cosmetic. Both ends of a range must have the same type, and without it the compiler cannot tell whether I meant a u32, an i64, or something else. So at is any index from zero to the maximum.

This raises a fair question. The generated integer can be as large as usize::MAX, about 1.8 billion billion on a 64-bit machine, while a short document has only a handful of positions. A five-byte string like "hello" has just six offsets, zero through five. So almost every raw value the generator produces lands far past the end of the buffer rather than inside it. Why draw from the whole usize range at all? The reason is that the three strategies are evaluated independently. The generator for at does not know how long the base string it is paired with will be, so it cannot produce an index that is valid for that particular document. The first line of the body deals with this.

let at = at % (doc.len() + 1);   // an offset into the buffer

The modulo maps any generated integer into the range 0..=doc.len(), where doc.len() is the length of the document in bytes. The range is inclusive at the top because you can insert at the end of the buffer as well as inside it. Generating a wide value and then mapping it into range is a common pattern with proptest, because it keeps every generated value usable instead of throwing away the ones that fall out of range. There is a more precise tool for producing an index into a generated collection, the prop_flat_map combinator, which lets one strategy depend on the value another produced. For a single index the modulo is simpler.

So the signature reads as follows: for any strings base and ins, and any index at mapped into range, the body must hold. proptest runs the body many times with different values, and the test passes only if the property holds for all of them.

When a property fails

Run this property as written and it does not reach the assertion. It panics, and not on "hello". It panics on an input I would not have typed into a test by hand.

A raw generated input is not much use as a bug report. If the framework told me the test failed on a four thousand character random string, that would be a puzzle to work through, not a defect I can read. What makes property-based testing practical is shrinking. When proptest finds a failing input, it does not report that input directly. It looks for the smallest and simplest input that still fails, using shorter strings, smaller indices, and values closer to zero, and reports that instead. The failure shrinks down to something like this.

minimal failing input:
    base = "é"
    ins  = ""
    at   = 1

panic: byte index 1 is not a char boundary

That is a report I can act on, and the surprise is where the panic comes from. It is not a bug in insert or delete. It is my own line, at % (doc.len() + 1). In Rust a string index is a byte offset, and len() returns a length in bytes, so the modulo gives me some offset between zero and the byte length. But not every byte offset is a place you are allowed to edit. An edit must land on a UTF-8 character boundary, and insert panics when it does not. The character é is two bytes, so the only valid positions are 0 and 2; the offset 1 falls inside the character. The generator produced the smallest string and index that break an assumption I had written into the test without noticing it: that any offset from zero to the length is a valid place to insert.

The fix is to generate only offsets that fall on character boundaries. Rust gives me the boundaries through char_indices, and I add the end of the string as the last position.

use proptest::prelude::*;

fn char_boundary_offsets(s: &str) -> Vec<usize> {
    s.char_indices()
        .map(|(i, _)| i)
        .chain(std::iter::once(s.len()))
        .collect()
}

proptest! {
    #[test]
    fn insert_then_delete_is_identity(base in ".*", ins in ".*", raw_at in 0usize..=usize::MAX) {
        let offsets = char_boundary_offsets(&base);
        let at = offsets[raw_at % offsets.len()];   // a valid edit position

        let mut doc = Document::from_str(&base);
        let before = doc.text().to_string();

        doc.insert(at, &ins);
        doc.delete(at, at + ins.len());

        prop_assert_eq!(doc.text(), before);
    }
}

There is a more general point here, beyond the Unicode detail. A property test does not only run more inputs than an example. It makes me say what a valid input is. Writing the example test, I never had to define the set of legal insertion points, because I only used position 5 of "hello", which happens to be legal. The property could not be written until I stated, in code, that a legal position is a character boundary. So the generator did not simply find a case I had forgotten. It exposed an assumption I had never written down, which is what happens when a rule has to hold for every input rather than for the one I picked.

The type system does not close this gap. String guarantees the buffer is valid UTF-8, but an index into it is a plain usize, and the type says nothing about whether that usize lands on a boundary. That check happens at runtime, and until I wrote the property it was happening nowhere in my tests.

Where property-based testing helps, and where it does not

It would be easy to read this and decide that property tests should replace example tests. They should not, any more than tests replace types. Each answers a different question, and property testing has a cost. A good property is harder to find than a good example, and a weak property is worse than no test at all.

The cost is in finding the rule. “For input "hello", expect "hello world"” takes a second to write. “For all documents and all edits, this relationship holds” takes real thought, and if the rule is slightly wrong the test is either flaky or empty. The common mistake is a vacuous property, one that cannot fail.

prop_assert!(doc.len() >= 0);       // len() is usize; always true

That runs a few hundred times, passes every time, and checks nothing. A property that is merely true is not the same as a property that constrains the code. The useful properties describe a relationship tight enough that a wrong implementation cannot satisfy it: round-trips such as decode(encode(x)) == x, idempotence such as f(f(x)) == f(x), order-independence where applying two edits in either order gives the same result, and conservation of some quantity such as length or character count. If I cannot name a relationship like that, an example test is the honest choice, and a property adds nothing.

Here is the rule I settled on. A property test earns its place when there is a rule that must hold across many inputs, not a single value I am checking. save then load should return the same buffer, and that is a rule. “The About dialog shows version 2.1” is a single value, and writing it as a property gains nothing.

Types, examples and properties

I now think of three tools rather than two, each covering what the previous one cannot.

Types rule out illegal shapes before the program runs. A Result that must be used, an Option that cannot be read without handling the empty case, a closed enum that forces every branch to be handled. The compiler settles these for every input at once, and this is the cheapest of the three.

Example tests check specific behaviour that the types allow but do not pin down. That inserting " world" at index 5 of "hello" gives "hello world" and not "worldhello" is a fact about values, and the type checker has no opinion on it. An example fixes one such fact.

Property tests cover the space in between, the behaviour that must hold across all the inputs the type still allows and that no single example can cover. The UTF-8 boundary case sits there. It is legal according to the type, missed by the examples, and wrong in general. The useful thing is to be clear about how behaviour will be evaluated before writing the code, which is the test-first instinct. A property is one clear way to state that evaluation, because it forces me to say what must always be true rather than what happened to be true the one time I ran the code.

The three fit together in order. Push what you can into types, so the compiler proves it for every input. Use examples to record the specific behaviour you have decided on. Use properties for the rules that must hold everywhere and that you would never cover by hand.

Summary

An example test checks the inputs I thought of. A property-based test states a rule and lets the framework generate inputs that try to break it. That is the difference that matters. On my own I tend to write the tests my code already passes, because I imagine the same cases when I write the code and when I test it. The generator does not share that imagination, and shrinking makes what it finds small enough to read and fix. Writing the property also forces me to say what a valid input is, which is often where the real gap turns out to be. The reason to reach for property testing is not that the code is otherwise untestable. It is that it exercises the inputs I would not have chosen, and in a text editor those are often the inputs a real user will produce.

References and Notes

Looking for alternatives, I came across a Git feature that I had heard about before but never used seriously: worktree. At first glance, worktree looks like a convenience feature. It allows multiple working directories to be attached to the same repository so that different branches can be checked out simultaneously. That is useful on its own, but what caught my attention was the implication behind it.

Most Git users, myself included, tend to think of a repository as a working directory that happens to contain a .git directory. Git does not view things this way. From Git’s perspective, the repository is the object database, and the working directory is one view into that repository. Worktree exists because Git was never fundamentally limited to a single working directory. Most of us simply use it that way.

The Mental Model Most of Us Carry

Consider a typical repository:

my-project/
├── src/
├── docs/
├── pom.xml
└── .git/

Most developers naturally view this as a single unit: a repository, a working directory, and a currently checked-out branch. That mental model is not wrong, but it is incomplete. The branch is not stored in the working directory. The commit history is not stored in the working directory. The object database is not stored in the working directory. Those things live inside .git, and the files visible in the working directory are a projection of a particular commit from the repository.

Once viewed from that perspective, an interesting question emerges: why should a repository be limited to a single projection? Git’s answer is that it isn’t.

Creating a Worktree

Suppose the current repository is checked out on a feature branch:

git switch feature/customer-notifications

A second working directory can be created using:

git worktree add ../customer-notifications-main main

Git creates a new directory alongside the existing one:

projects/
├── customer-notifications/
└── customer-notifications-main/

The first directory remains on the feature branch, and the second is checked out on main. Both can be opened in separate terminals, editors, or IDE windows, and changes in one working tree do not affect the other. From a developer’s perspective, it feels almost like having two clones, except that Git is not creating another repository.

Looking at What Git Actually Creates

After creating a worktree, the repository structure changes slightly. Inside .git, a new directory appears:

.git/
├── objects/
├── refs/
├── logs/
└── worktrees/

Listing the contents of .git/worktrees/ reveals a subdirectory for each additional worktree, containing files such as HEAD, gitdir, commondir, and index. The exact contents may vary across Git versions, but the important observation is that Git is maintaining metadata for an additional working tree while continuing to share the underlying repository. This is why creating a worktree is extremely fast. Git is not duplicating the commit graph, objects, tags, or references. It is creating another working directory and associating it with the existing repository.

Shared and Separate State

The easiest way to understand worktrees is to identify what is shared and what is independent. The object database, branch references, tags, hooks, and configuration are all shared across worktrees. Each worktree has its own working directory contents, its own index, and its own HEAD pointing somewhere into the shared refs/heads/. This separation is what allows two branches to be active simultaneously without two repositories, and it has a few practical consequences worth noting.

A git fetch in any worktree updates the shared refs, so the new commits are immediately visible from every worktree attached to the repository. Hooks installed in .git/hooks/ run regardless of which worktree triggered them, which catches some people out when they expect per-worktree hook behavior. And because branch refs are shared, a branch checked out in one worktree cannot be checked out in another, which is the next thing worth looking at.

Why Git Refuses Certain Operations

Suppose feature/customer-notifications is already checked out in one worktree. Attempting to check it out in another produces an error:

$ git worktree add ../another-copy feature/customer-notifications
fatal: 'feature/customer-notifications' is already checked out at '/path/to/customer-notifications'

This can initially seem restrictive, but the restriction makes sense given the shared-ref design. A branch is a movable reference, and if two working directories could manipulate the same branch independently, Git would have no way to determine the intended state of that reference. The restriction prevents ambiguity and protects branch state from concurrent updates.

Worktree Versus Stash

Worktree and stash solve different problems, even though both come up when someone needs to switch context briefly. A stash temporarily preserves uncommitted changes so another task can be performed in the same working directory, while a worktree creates an entirely separate workspace. If I need to quickly pull main, perform a rebase, and continue working, a stash is reasonable. If I expect to spend hours investigating another branch while keeping the current work untouched, a worktree is the better choice. The distinction is subtle but important: a stash preserves state, while a worktree preserves context.

Worktree Versus Another Clone

Historically, some developers solved this problem by maintaining multiple clones of the same repository. That approach works, but every clone maintains its own object database, references, and repository metadata, which means fetches must be performed in each clone separately and disk usage grows linearly with the number of copies. Worktrees share those resources. For smaller repositories the difference is negligible, but for larger repositories containing years of history it becomes noticeable. More importantly, worktrees communicate intent. They represent multiple views into the same repository rather than multiple copies of the repository itself.

The Bare Repository Approach

One workflow worth mentioning treats worktrees as the primary interface rather than an occasional convenience. The repository is cloned bare, and every branch the developer wants to work on becomes its own worktree:

git clone --bare git@github.com:example/project.git project.git
cd project.git
git worktree add ../project-main main
git worktree add ../project-feature feature/customer-notifications

There is no “main” working directory in this model. Every working directory is a worktree, and the bare repository at the root contains only the shared state. Some developers organize their worktrees inside a .worktrees/ subdirectory of the project to keep things tidy. Whether this layout is worth adopting depends on how often parallel branches are needed, but it makes Git’s underlying model unusually visible.

Managing Worktrees

Git provides a few commands for managing worktrees. git worktree list shows all active worktrees and the branches they point to:

/home/palak/projects/my-project          a1b2c3d [feature/customer-notifications]
/home/palak/projects/my-project-main     e4f5g6h [main]

git worktree remove <path> removes a worktree. If a worktree directory is removed manually outside of Git, the stale metadata can be cleaned up with git worktree prune. Most day-to-day usage rarely requires anything beyond these commands.

Final Thoughts

Before learning about worktrees, I implicitly assumed that a Git repository and a working directory were inseparable concepts. Git itself makes no such assumption. A repository is an object database containing commits, trees, blobs, references, and metadata, while a working directory is one representation of repository state at a particular point in time. Viewed from that perspective, worktrees feel less like an advanced feature and more like a natural consequence of Git’s architecture.

The feature is useful for avoiding branch switching, preserving context, and keeping parallel streams of work isolated. But the more interesting lesson is the one it reveals about Git itself. Many of the limitations we assume are built into our tools are actually limitations of the mental models we carry around. Git worktree is a small reminder that those two things are not always the same.

These are some of the notes.

A green test is not a correct program

The function performed following tasks - write the buffer to a path, then mark the document as having no unsaved changes.

pub fn save(&mut self, path: &Path) -> io::Result<()> {
    std::fs::write(path, self.text());   // the Result here is silently dropped
    self.modified = false;
    Ok(())
}

and I had also written the following test:

#[test]
fn save_writes_contents_and_clears_modified() {
    let path = std::env::temp_dir().join("save_test.txt");
    let mut doc = Document::from_str("abc");
    doc.insert(3, "def"); // buffer is now "abcdef", with unsaved changes

    doc.save(&path).unwrap();

    assert_eq!(std::fs::read_to_string(&path).unwrap(), "abcdef");
    assert!(!doc.is_modified()); // a successful save clears the dirty flag

    let _ = std::fs::remove_file(&path);
}

It built a document, made a change, saved it to a temporary file, read the file back, checked that the contents matched, and confirmed the document no longer reported unsaved changes. The test passed, which would normally be the end of it, except that the implementation had a bug on the first line.

std::fs::write returns a Result, because writing to disk can fail in any of the ordinary ways: the disk fills up, the directory does not exist, the process lacks permission. The function ignored that return value, so a failed write would still fall through to self.modified = false and then Ok(()). The editor would report the file as saved, clear the unsaved-changes marker, and lose the user’s work, which is the one bug a text editor cannot ship. The test stayed green the whole time because it only ran the happy path, and a passing happy-path test says nothing about the behaviour on the failure paths it never visits.

The compiler reviews shape, not intent

I missed the bug, but the compiler flagged it:

warning: unused `Result` that must be used

Rust marks Result as #[must_use]¹, so the lint fires when a fallible call’s return is dropped. The warning is real help, but it is still only a warning. I could have written this instead and quieted it:

let _ = std::fs::write(path, self.text());   // explicitly discard it
self.modified = false;
Ok(())

That compiles cleanly. The type checker is satisfied: the signature returns a Result, a Result is returned, the borrow rules² hold, the shape of the program is right. What the compiler has no way of checking is what I meant by it – that a failed write must not clear the modified flag, that it must not be reported as success.

This is where the work of compiler stops. It checks that the types line up, that fallibility is visible in signatures, that a Result is not accidentally ignored, that the borrows are valid; it does not check that the program does the right thing with any of those things once it has them.

A regression test only counts once you have seen it fail

The fix is one character, the ? operator, which turns the dropped result into a propagated one:

std::fs::write(path, self.text())?;
self.modified = false;
Ok(())

If the write fails, the function exits before clearing the modified flag. I had no test that would have caught the bug. Adding ? without adding a test would leave me trusting the code for no reason at all, so I wrote the test that should have existed from the start.

#[test]
fn failed_save_surfaces_error_and_keeps_modified() {
    let mut doc = Document::from_str("important data");
    doc.insert(0, "!"); // the document now has unsaved changes

    // a directory that does not exist, so the OS write is forced to fail
    let bad_path = std::env::temp_dir().join("no_such_dir").join("file.txt");
    let result = doc.save(&bad_path);

    assert!(result.is_err());     // the failure must reach the caller
    assert!(doc.is_modified());   // and the buffer must still be dirty
}

Then I put the bug back, ran the suite, watched this test go red, restored the fix, and watched it go green. Putting the bug back to confirm the test fails for the right reason sounds like ceremony but is not. A regression test you have never seen fail is one you trust by faith, and the first section already showed how little a passing test is worth on its own.

Types and tests answer different questions

A type system is good at making certain kinds of lies impossible: a value that may be absent cannot pretend to be present, a fallible operation cannot pretend to be infallible, a closed set of cases cannot pretend one branch does not exist. That is powerful, but it is structural. Rust can make the failure visible, warn me when I accidentally ignore it, and force the code to admit that saving may fail, but it cannot encode the editor’s rule that if saving fails, the document must remain dirty. That rule lives at the level of behaviour, not shape.

You can push more behaviour into types than that suggests. The typestate pattern³ encodes a rule into a type, so that an illegal operation does not compile instead of failing a test. The save bug can be narrowed this way by making the clearing of the modified flag depend on a value that only a successful write can produce:

struct Saved;   // nothing outside this module can construct one

fn write(&self, path: &Path) -> io::Result<Saved> { /* ... */ }
fn mark_clean(&mut self, _proof: Saved) { self.modified = false; }

Now mark_clean cannot be called without a Saved, and the only thing that hands one back is a write that returned Ok, so the exact bug from the first section is harder to write by accident.⁴ The boundary between what a type can prove and what only a test can prove is not fixed; it moves with how much you choose to encode.

What stays out of reach, at least in Rust, is value-level correctness, which is the relationship between specific input and output values. That needs dependent types, which Rust does not have. The type checker has no opinion on whether inserting " world" at index 5 of "hello" gives "hello world" or "worldhello"; both are valid Strings, and which one you meant is a question only a test can answer.

The instruction that falls out of this is short: do not test what the type system has already made impossible, and do test the behaviour the type system is silent about. A small example came up in the next step. A document needs to remember the file it came from, but a new, untitled buffer has not come from anywhere, so the honest type for that field is Option<PathBuf>, and Rust will not let you read the path without handling the case where there is none. The “I forgot to check for the missing file” bug is not caught by a test here; the program that contains it does not compile, and writing a test for it would only re-prove what the compiler has already proven.

“Test everything that can fail” is too blunt

I used to carry a simple rule: every operation that can fail gets a failure test. It made sense in the Java world I grew up in, and after the save bug I wrote one for open too – a test asserting that opening a missing file returns an error. Then I deleted it.

The save failure test is worth keeping, but not because saving can fail. It is worth keeping because save changes state on the way to succeeding: it clears the modified flag. The assertion in that test that actually matters is not assert!(result.is_err()) but assert!(doc.is_modified()), which guards the state that a failed save could otherwise leave wrong. open is different; it builds a value and returns it, changing nothing and leaving nothing wrong when it fails. The only thing a failure test there could check is whether I propagated the error instead of swallowing it, which is thin, with no state behind it to protect.

So the rule sharpened into something narrower: a failure test earns its place in proportion to the state a failure could leave wrong, and where failure only propagates cleanly there is little for it to guard. That is not an argument for testing less out of laziness, but for putting tests where things can actually break instead of spreading them evenly out of habit.

The point

This is not an argument against tests or against types, but for knowing which of the two you are leaning on at any given moment. Lean only on tests and you will keep writing assertions to re-establish guarantees a good type system would give you for free, at compile time, for every input. Lean only on types and you will ship code that is well-formed and quietly wrong: green in every structural sense, still losing files. The healthier division is to let the type system carry what it can – null-safety, exhaustiveness, fallibility made visible, and as much of your state machine as you are willing to encode – and to spend your tests on the behaviour that is left over.

That line moves. Typestate carries more than people expect, and dependent types would carry more again, but wherever you draw it for a given program the rule is the same: do not ask a type to prove what only a test can, or a test to re-prove what the type already guarantees. How far the line can move also depends on the language. Rust leans on two guarantees that Java’s type system holds you to less strictly. The first is that absence has a single shape: Rust has no null, so a value that might not be there is an Option and the compiler will not let me touch it without handling the empty case, whereas Java’s Optional⁵ sits beside null rather than replacing it and leaves .get() available to learn my mistake at runtime. The second is move semantics: Java can make a proof-of-save exist through a private constructor and a factory, but it cannot make it spent, so nothing would stop me clearing the flag with the same token twice; Rust takes the value away when it is used, and that is the part Java has no equivalent for. In both cases the work does not vanish – it moves from the compiler to my tests and my discipline. The reason to reach for Rust here is not that the editor is otherwise impossible, which it plainly is not, but that a more expressive type system settles more of the program’s correctness before a single test runs.

References and Notes

This behavior causes some interesting bugs and awkward APIs. We will explore some of them in this post.

1. List<String>.class does not exist

We are all familiar with List.class, String.class, Integer.class but List<String>.class is an illegal construct in Java. At runtime, Java does not have a separate class object for List<String> versus List<Integer>.

That is why APIs that often accept this Class<T> type break down for generic types.

For example, this API looks clean:

<T> T read(String json, Class<T> type)

It works well for a normal object:

User user = read(json, User.class);

However, the following will be impossible:

List<User> users = read(json, List<User>.class);

There is no List<User>.class to pass. You may be tempted to make it work by trying:

List<User> users = read(json, List.class);

But now the runtime only knows that the target type is List. It does not know that the list is supposed to contain User objects.

This is where bugs begin. A JSON library, for example, may deserialize the JSON array into a List<LinkedHashMap> instead of a List<User>. The code may compile, but fail later when you try to use one of the elements as a User.

List<User> users = read(json, List.class); 

User user = users.get(0); // may fail at runtime

The problem is not that the list may be empty. That would be a normal collection issue. The type erasure problem is that even if the list contains elements, those elements may not be of the type the source code appears to promise.

To solve this, libraries invented alternate ways to carry generic type information into runtime APIs.

Jackson uses TypeReference<T>:

List<User> users = objectMapper.readValue(json, new TypeReference<List<User>>() {});

Gson uses TypeToken<T>:

List<User> users =
    gson.fromJson(json, new TypeToken<List<User>>() {}.getType());

Spring uses ParameterizedTypeReference<T>:

ResponseEntity<List<User>> response =
    restTemplate.exchange(
        url,
        HttpMethod.GET,
        null,
        new ParameterizedTypeReference<List<User>>() {}
    );

These APIs look awkward because they are working around the same missing runtime concept: a class-like object that represents a fully parameterized generic type. If Java supported class literals for parameterized types such as List<User>.class, many of these APIs could have been simpler.

2. instanceof List<String> does not work

Another place where type erasure shows up is runtime type checking. In Java, we commonly use instanceof to check the type of an object.

if (obj instanceof String) {
...
}

But Java won’t let you write

if (obj instanceof List<String>) {
 ...
}

The reason is that List<String> is not fully available at runtime. After type erasure, the runtime can check whether the obj is a List, but it cannot directly check whether it is specifically a List<String>.

So, Java only allows this:

if (obj instanceof List<?>) {
...
}

This tells us that obj is some kind of List. It does not tell us what kind of elements the list contains and we must inspect each element individually, if we really care:

if (obj instanceof List<?> list &&
    list.stream().allMatch(String.class::isInstance)) { //this check returns true for an empty list as well
     ...
}

This works, but it is much more verbose than a normal runtime type check. It also changes the nature of the check. We are no longer asking the JVM, “Is this a List<String>?” We are asking, “Is this a List, and do all of its current elements happen to be strings?”

3. Method overloads can clash after erasure

Java lets you overload methods when their parameters are different. For example, this is fine:

void process(String name) {...}

void process(Integer id) {...}

At runtime these are still different. However, with generics, method overloading gets a little tricky.

You may think that this will work:

void process(List<String> names) {...}

void process(List<Integer> ids) {...}

But Java rejects this because after type erasure both effectively are the same method:

void process(List names) {...}
void process(List ids) {...}

Both have the same erased signature void process(List). The compiler sees the method collision and throws this error:

name clash: process(List<Integer>) and process(List<String>) have the same erasure

The workaround usually is to give methods different names processNames(List<String> names) and processIds(List<Integer> ids).

Another option is to introduce wrapper types:

record Names(List<String> names) {}
record Ids(List<Integer> ids) {}

void process(Names names) {...}

void process(Ids ids) {...}

This works because Names and Ids are real runtime types. They survive erasure, unlike List<String> and List<Integer>.

4. Generic arrays are painful

Type erasure also makes arrays and generics uncomfortable together.

In Java, arrays know their component type at runtime. For example, a String[] knows that it is an array of String. If you try to put an Integer into it, the JVM can detect the problem and throw an ArrayStoreException.

String[] names = new String[10];
Object[] values = names;

values[0] = 42; // ArrayStoreException at runtime

Generics work differently. A List<String> does not carry String as a full runtime type in the same way. After erasure, it is mostly just a List.

That mismatch is why Java does not allow this:

T[] values = new T[10]; // illegal

The runtime does not know what T really is, so it cannot create an array with the correct component type.

The workaround is to pass the component type explicitly:

static <T> T[] createArray(Class<T> type, int size) {
    return (T[]) java.lang.reflect.Array.newInstance(type, size);
}

Usage:

String[] names = createArray(String.class, 10);
Integer[] ids = createArray(Integer.class, 10);

The generic type T is known to the compiler, but not enough is available at runtime. So the API asks the caller to pass a Class<T> token manually.

This is another cost of type erasure. The type appears obvious in the source code, but runtime code still needs an explicit type descriptor to do the right thing.

Why these workarounds work

Type erasure is not total. Java still records generic type information in the class file, in a metadata section called the Signature attribute. This is kept for declarations: fields, method parameters and return types, and a class’s generic superclass and interfaces. What gets erased is the type of a value at runtime. A List<String> object and a List<Integer> object share the same List.class, so an object cannot tell you its element type. But a declaration can.

That retained information is readable through reflection. Field.getGenericType(), Method.getGenericReturnType(), and Class.getGenericSuperclass() return a java.lang.reflect.Type, which can be a ParameterizedType such as List<User> rather than a plain List.

This is the trick behind TypeReference, TypeToken, and ParameterizedTypeReference. Writing new TypeReference<List<User>>() {} creates an anonymous subclass, and its generic superclass TypeReference<List<User>> is a declaration. So List<User> is preserved in that subclass’s metadata, and the library recovers it with getClass().getGenericSuperclass():

Type type = new TypeReference<List<User>>() {}
    .getClass()
    .getGenericSuperclass(); // ParameterizedType: List<User>

Conclusion

Once your code crosses a runtime boundary, assume the generic type will not be there. Design the API to carry the type explicitly, through a Class<T> token, a TypeReference<T>, a wrapper type, or a type tag you store in the data yourself, rather than trusting that the compiler’s view of the type survives into runtime.

I wanted to do this without losing my existing shell configuration, installed tools, IDE state, and Claude Code session history.

It turns out a complete rename is very doable. The data-loss fear is mostly misplaced. The real risk is not deleted files, but stale hard-coded paths. In this post, I cover exactly how I did it.

A Little Gotcha Before We Begin

usermod refuses to touch an account that is logged in or has running processes:

usermod: user old-user is currently used by process NNNN

Since this was the only admin account on the machine, I couldn’t rename it while logged into it. The fix is a throwaway second admin account that does the surgery while the real account is fully logged out.

With that, here are the steps to change the identity of an existing user.

Step 1 — Create a temporary admin

While still logged in as the old user:

sudo adduser tempadmin
sudo usermod -aG sudo tempadmin

Step 2 — Log out completely and switch

This part matters: Log Out of GNOME — not lock, not “Switch User”. The old session must actually end. Then log in as tempadmin at the greeter and open a terminal.

Step 3 — Confirm the old account is truly idle; if needed terminate the sessions

who                                   # old user should not appear
pgrep -u old-user                     # should print nothing
sudo loginctl terminate-user old-user 2>/dev/null
pgrep -u old-user || echo "clear - safe to proceed"

Proceed to Step 4 if the message says clear - safe to proceed.

Before renaming, record the account’s current numbers so you can confirm nothing shifted afterward — and so you’re not assuming mine:

id old-user        # note the uid=, gid=, and the full groups= list

On my machine both the UID and GID were 1000 (the first account the Ubuntu installer creates). Yours may differ — on a multi-user box, a migrated system, or one with service accounts, the first human user isn’t always 1000. Whatever your values are, they should be identical before and after; keeping them unchanged is exactly why we never pass -u.

Step 4 — The actual rename

Three commands do the core work:

# Rename the login AND move/rename the home dir (last arg is the CURRENT name):
sudo usermod -l systemhalted -d /home/systemhalted -m old-user

# Rename the matching private group; the GID stays unchanged:
sudo groupmod -n systemhalted old-user

# Update the display/full name shown at the login screen:
sudo usermod -c "systemhalted" systemhalted

Why this is safe: the UID stays unchanged, 1000 on my machine — usermod -l changes the login name and -d -m moves the home directory’s contents, but the UID stays put because I’m not passing -u. With the numeric owner unchanged, usermod -m moves the home directory and tries to adapt ownership, permissions, ACLs, and extended attributes under the new path — though the manpage notes some cases may still need manual fixing. In the normal same-UID case, the intent is relocation rather than deletion, but I still treated backup as mandatory. The account’s authentication entry is preserved through the rename, so the same password keeps working.

Step 5 — Fix what usermod doesn’t touch

This is where the real work hides. usermod renames the account, but several things keep pointing at the old path or the old name.

(a) Hard-coded paths. A scan of my home turned up ~196 files under .config, .local, and .claude containing the literal string /home/old-user — almost all of it, in my case, JetBrains and VS Code state. Editing them in bulk is a trap: many are binary or fragile JSON, and a stray sed can corrupt them. The clean, reversible fix is a single compatibility symlink:

sudo ln -s /home/systemhalted /home/old-user

Now every stale /home/old-user/... reference resolves transparently. No file edits, no corruption risk. It’s not entirely free, though — a lingering symlink can confuse backups, scripts, future users, or security scans — so keep it (documented) until you’re confident nothing references the old path, then remove it.

(b) The user crontab spool isn’t auto-renamed (the same goes for at jobs, if you use them):

f=/var/spool/cron/crontabs/old-user
sudo test -f "$f" && sudo mv "$f" /var/spool/cron/crontabs/systemhalted \
  && sudo chown systemhalted:crontab /var/spool/cron/crontabs/systemhalted

(c) GNOME’s AccountsService keeps per-user login-screen prefs keyed by name:

f=/var/lib/AccountsService/users/old-user
sudo test -f "$f" && sudo mv "$f" /var/lib/AccountsService/users/systemhalted

(d) Claude Code session history. Claude Code names its project folders after the working directory. Mine were all -home-old-user-*, so prior session history was keyed to the old path:

cd /home/systemhalted/.claude/projects

for d in -home-old-user*; do
  mv "./$d" "./${d/-home-old-user/-home-systemhalted}"
done

Two gotchas bit me here. First, the directory names start with -, so every tool treats them as options — prefix paths with ./ (or use --) or you’ll get a wall of invalid option -- 'h'. Second, a -home-systemhalted folder already existed from a session run after the home move, so the bare rename would collide. I merged that one instead of moving it:

cp -a --update=none "./-home-old-user/." "./-home-systemhalted/" \
  && rm -rf "./-home-old-user"

--update=none means any file that already exists at the destination is left untouched — it’s never overwritten, regardless of timestamps.

Step 6 — Verify before trusting it

getent passwd systemhalted    # home, shell, GECOS all correct
getent group  systemhalted    # same group, GID unchanged (1000 on mine)
id systemhalted               # the important one
ls -ld /home/systemhalted     # owned systemhalted:systemhalted
ls -ld /home/old-user         # symlink -> /home/systemhalted

The line I cared most about was id, confirming the renamed account was still in both sudo and docker — i.e. admin and container access carried over intact.

Step 7 — Reboot and live in it

sudo reboot

Log in as systemhalted with the old password. Terminal opens in /home/systemhalted, the IDEs launch with their state, docker ps works, my shell tooling works, and Claude Code shows all the prior history.

Step 8 — Remove the temporary admin

Only after the renamed account is confirmed working:

sudo deluser --remove-home tempadmin

Worth knowing: if deluser ever runs without --remove-home (or partially), it leaves two orphans behind — /home/tempadmin and /var/lib/AccountsService/users/tempadmin — which you then clean up by hand:

sudo rm -rf /home/tempadmin
sudo rm /var/lib/AccountsService/users/tempadmin

Rollback, just in case

Everything is reversible from the tempadmin session until you delete it. If anything looked wrong, I had this ready:

sudo rm /home/old-user
sudo usermod -l old-user -d /home/old-user -m systemhalted
sudo groupmod -n old-user systemhalted
sudo usermod -c "Old User" old-user

Important Notes and Observations

• No data was lost in my case — a same-UID rename moves files; it doesn’t delete them by design.
• The danger is stale paths, not lost files — and one symlink neutralizes all of them.
• Never bulk-sed IDE state — it’s binary/JSON and corrupts easily.
• Secrets mostly survive, but check them. GNOME Keyring is unlocked by your login password, which didn’t change, so keyring-backed secrets keep working (maybe a one-time unlock prompt). Saved Wi-Fi depends on setup — NetworkManager profiles may be system-wide, per-user, or keyring-backed — so verify rather than assume.
• The compatibility symlink is practical, not free. It saves you from editing hundreds of files, but document it and remove it once you’re sure nothing uses the old path.

The whole thing took one reboot and zero reinstalls. The account that started life as old-user is now systemhalted as far as normal desktop and shell tooling are concerned. (The kernel itself never cared — to it, credentials are numeric UIDs and GIDs, not names; the login name was only ever a label on top of UID 1000.)

Disclaimer: This worked on my single-admin Ubuntu/GNOME machine, and I’m sharing it as a record of what I did — not as a guaranteed recipe for yours. Renaming a user touches login, ownership, and system state, so treat every command here as something to understand before you run it, not to paste blindly. Setups differ (other desktops, network/LDAP accounts, encrypted or NFS home directories, services running as the user), and any of those can change the outcome. Take a backup first, keep the tempadmin escape hatch until you’ve confirmed everything works, and proceed at your own risk. The username here (old-user) is a placeholder; the real one has been obfuscated.

The structure, if you can call it that, was two systems running side by side and neither of them working. I had read about Zettelkasten somewhere, read a book about it, watched some videos, and set up the standard folders — Inbox, Input, Output, Zettel — without ever using them. Alongside that I had an older hierarchy of Work, Personal, and Writing, where I was actually saving things, more or less at random. Whichever folder I thought of first got the note. I had been telling myself this was fine because the notes were captured, and search would do the rest. Search did not do the rest. Most of the time I did not know what to search for.

Why I finally bothered

I am an engineering manager. In a normal week I need notes from team 1:1s, architecture decisions, books I am reading, an AI/ML study plan I run at weekends, ideas for writing, astrophotography logs, and guitar stuff. None of that is unusual. What forced the cleanup was that several times in a couple of months, I needed a specific note before a meeting and could not find it in time.

The problem was not that I had too many notes. It was that I had never decided what the system was supposed to do.

Simplification

One Saturday I sat down and wrote out what I actually use notes for: work, personal life, things I am learning, writing I am working on, a daily journal, random ideas, and an archive for things I am done with. That came out as eight buckets, which I numbered so they would sort in the order I wanted:

00 Inbox
10 Work
20 Personal
30 Learning
40 Writing
50 Journal
60 Ideas
99 Archive

Each has a few subfolders. Work has Team Management, Meetings, Architecture, and Decisions. Learning has Books, Articles, Videos, Courses, and a Subjects folder where I keep my own synthesis on topics I am studying over time. Writing splits into Ideas, Drafts, Articles, Poetry, Stories, Books, and Published. The old Zettelkasten folders went into Archive. I was not using them, and pretending I would was part of why the system was a mess.

I learned later that what I had built was close to a combination of two methods I had not heard of when I started. PARA (Projects, Areas, Resources, Archives) organises by how actionable something is, not by topic. Johnny Decimal is a numbering scheme that gives every folder a permanent ID. People combine the two, and what I had was a rough version of that combination. It was nice to know but did not change anything. The structure worked because it matched how I think, not because it had a name.

The rule that fixed the overlap

The other thing I had been doing wrong, which the cleanup made obvious, was mixing notes and tasks. Things that belonged in Reminders were living in Apple Notes, and the other way around, and I was sometimes updating both for the same thing without realising it.

The rule I went with is simple. If it has a “done” state, it goes in Reminders. If it does not, it goes in Notes. Tasks, follow-ups, and things to keep an eye on go to Reminders; knowledge, drafts, and references go to Notes. It sounds obvious written down, but I had not been doing it. Once I started, the overlap was gone, and so was most of the duplication that came with it.

What I’d actually tell someone

The cleanup itself was not the hard part. The hard part was admitting that the Zettelkasten folders I had set up years earlier were aspirational, not active. I was impressed by how useful it was for Niklas Luhmann and thought it would be useful for me as well in the same way it was for him. However, these folders had been sitting there making the system look organised while making the actual mess worse.

Most of the work was being honest about what I really do with notes — capture things, look them up later, occasionally write something longer — and building the smallest structure that supported those three things. No folders that existed only because some method I had read about said they should.

If you are sitting on your own version of a thousand unfiled notes, do not go looking for a method to copy. Write down what you actually use notes for, build the folders that match, and leave room to fix it later. The other thing — and I am saying this because I ignored my own advice for years — is that half an hour once or twice a week spent clearing out the Inbox goes a long way. The cleanup itself is a one-time event. Keeping it clean is not.

What stood out to me, reading through it, was less the model architecture than the engineering discipline behind the choice of where to use which kind of model. The authors did not, in fact, reach for an LLM as the answer to every problem, which is increasingly unusual in this space. Instead, they appear to have asked, at each stage of the pipeline, what the cheapest model that does the job correctly actually is, and to have used that. Two of the numbers in the paper are particularly striking:

• An auto-encoder handles intent detection in 0.009 seconds instead of 1.6 seconds.

• A fine-tuned BERT model handles routing in 0.3 seconds instead of ~2 seconds.

The LLM, in their architecture, only enters the picture once this fast triage stage has narrowed the problem space — at which point it is being used for what large language models are actually good at, which is reasoning over a focused problem and generating narrative explanations for a human reader.

They also, sensibly, avoid the naive Text-to-SQL approach that has become almost a cliché in this area. Instead, they describe an augmented querying flow that brings business context, internal APIs, and a plan-and-execute style of decomposition to bear on the question. The system breaks a query into a sequence of steps, fetches the right data at each step, and produces explanations that a human reader can actually act on, rather than a single SQL query that is correct in the abstract but unhelpful in context. In their reported evaluation, the system holds 90th-percentile latency under roughly 13 seconds, with about 89% relevance and correctness as judged by human raters — which, for an agentic pipeline of this kind, is a meaningfully better number than I would have expected.

The most important idea in the paper, however, is, I think, the one that follows from all of this rather than being stated outright. It is the implicit observation that, in the model described, the user is no longer expected to navigate a dashboard at all. They are expected, instead, to ask questions of a system that already understands the schema, the metrics, the seasonality of the business, and the internal vocabulary that the team uses to talk about what is happening — and to receive, in response, an explanation of what happened and, where the data supports it, why it happened.

This is not, to be clear, an argument for replacing BI teams or data engineers; on the contrary, it is an argument for amplifying them. What this kind of system promises, if it works at the scale claimed, is to turn the underlying data infrastructure into something a non-specialist part of the business can actually query in plain language, and to receive a structured explanation back rather than a chart that someone still has to interpret. The future of analytics, on this view, is less likely to be a continued proliferation of dashboards and more likely to be a steady shift toward conversational understanding layered on top of structured data — with dashboards retained, in the end, mostly for the cases in which a human still wants to look at the underlying time series themselves.

If you are working on analytics platforms, on GenAI agents, or on data products of any meaningful scale, I would encourage you to read the paper end to end. It is, by some distance, the most thoughtful treatment of where to put which kind of intelligence in an agentic data system that I have come across in the last year.

References

In Part 7 of this series, we looked at summation error and how Kahan compensation can keep a long reduction from quietly losing low-order bits. This post is about a different and, in some ways, more consequential precision trap: the moment at which a value is rounded down to fewer digits in order to be displayed, stored, or reported.

Rounding looks like a small detail in the code, and it almost never is one. It is the place where many otherwise-correct calculations end up disagreeing with audits, ledgers, and reconciliations, often in ways that are difficult to diagnose because the disagreement only appears at scale. Most teams I have seen pick a rounding mode in roughly the same way that most people pick a film on a streaming service — by going with whichever option happened to be already playing — and in Java, the option that happens to be already playing is very often HALF_UP, on the implicit assumption that this is “normal” rounding.

“We’ll use HALF_UP. That’s normal rounding. Done.”

Then a pricing engine ships, or a statement generator, or a ledger, or a tax calculator, or an amortization schedule — and at some point afterwards it becomes apparent that rounding is policy, not arithmetic. Or, more precisely, it is arithmetic with consequences, and the choice of which rounding mode to apply turns out to be a product decision wearing mathematical clothing. This post is about how to make that choice deliberately, and about why HALF_UP is not the universal solvent it is so often assumed to be.

Quick Reference: Rounding Mode Cheat Sheet

Key rules:

• Avoid rounding double when policy matters: use BigDecimal from strings → The Classic Trap
• Decide when to round, not just how → Rounding Timing
• For money, use scaled integers internally → Money Pattern

The real problem: ties (the 5s)

The drama in rounding, almost without exception, is not really about choosing between 2.341 and 2.34. It is about ties — values that lie exactly halfway between two representable steps at the precision you are rounding to.

At two decimal places, the troublesome values are the ones that look like this:

• 1.005
• 2.675
• 10.125

It is worth being precise about what counts as a tie, because the term is sometimes applied loosely to values that are not, strictly speaking, halfway between anything. A value is a tie at a given scale only when the first discarded digit is exactly 5 and all subsequent discarded digits are exactly 0. So 1.005 is a tie at two decimal places, and so is 1.00500…, but 1.0051 is not a tie — it is, on inspection, closer to 1.01 than to 1.00, and any reasonable rounding mode will produce 1.01 regardless of how it handles ties.

The reason ties matter so much in practice is that, depending on which way the rounding mode resolves them, the cumulative effect across many operations can be quite different. If you consistently push ties in one direction — say, always upward — you introduce a systematic bias into every aggregate that depends on those rounded values. Sometimes that bias is exactly what the domain wants (retail prices, for instance, often round in a particular direction by convention); often it is not, occasionally it is actually prohibited by regulation or contract, and even when it is none of those things, it is the kind of pattern that tends to make reconciliation teams unhappy when the totals begin to drift.

Java’s rounding toolbox

Java’s standard library, in java.math.RoundingMode, exposes a fairly complete set of rounding policies. The interesting question, in the end, is rarely which of them is “available” — they all are — but rather which of them matches the contract the domain actually wants enforced. The available modes are:

• HALF_UP - round to nearest; if exactly halfway, round away from zero (1.5 -> 2, -1.5 -> -2)
• HALF_DOWN - round to nearest; if exactly halfway, round toward zero (1.5 -> 1, -1.5 -> -1)
• HALF_EVEN - round to nearest; if exactly halfway, round to the result whose last kept digit is even (banker’s rounding, also called round-half-to-even)
• UP - always away from zero
• DOWN - always toward zero (truncate)
• CEILING - toward positive infinity (-1.231 -> -1.23, 1.231 -> 1.24 at 2dp)
• FLOOR - toward negative infinity (-1.231 -> -1.24, 1.231 -> 1.23 at 2dp)
• UNNECESSARY - throw if rounding would be required (which I have come to think of as a “make bugs loud” mode)

It is also worth recalling that calling setScale(...) without specifying a rounding mode will, if rounding turns out to be required, throw — which is a sensible default but is occasionally surprising:

new BigDecimal("1.234").setScale(2); // throws ArithmeticException

And the most important practical rule, which I find myself repeating in code review more often than any other rule in this area:

If you care about exact decimal policy, avoid rounding a double. Round a BigDecimal created from a string (or from an exact integer scale).

The classic trap: new BigDecimal(double)

This is the trap that catches almost everyone at least once, and it is worth showing in code rather than just describing in prose:

import java.math.BigDecimal;
import java.math.RoundingMode;

public class RoundingTrap {
  public static void main(String[] args) {
    BigDecimal a = new BigDecimal(1.005);               // from double
    BigDecimal b = new BigDecimal("1.005");             // from string

    System.out.println(a.setScale(2, RoundingMode.HALF_UP)); // 1.00
    System.out.println(b.setScale(2, RoundingMode.HALF_UP)); // 1.01
  }
}

The expectation, of course, is that both values round to 1.01. What actually happens is that the first one rounds to 1.00, because 1.005 as a binary floating-point value is not, in fact, exactly 1.005 — it is a hair less than that, by exactly the small amount that binary floating point cannot quite represent the decimal fraction 0.005. When you then ask BigDecimal to round that approximate value, it dutifully rounds the value it actually has, which falls just below the tie boundary.

If you find yourself, in a finance system, looking at a result in which 1.005 appears to round to 1.00 under HALF_UP, the right interpretation is not that Java is being eccentric. The right interpretation is that you have fed an approximate binary representation into an exact decimal rounding routine, and that the routine has, accurately and unhelpfully, given you the answer that corresponds to the value it was actually given.

What about BigDecimal.valueOf()?

There is a third constructor in this space which sits between the two previous options and which readers trip over often enough that it is worth treating explicitly: BigDecimal.valueOf(double).

BigDecimal a = new BigDecimal(1.005);        // Dangerous: uses exact binary representation
BigDecimal b = BigDecimal.valueOf(1.005);    // Safer: uses Double.toString() internally
BigDecimal c = new BigDecimal("1.005");      // Safest: exact decimal from string

BigDecimal.valueOf(double) will, in most everyday cases, behave as if you had used the string constructor, because it routes the conversion through Double.toString() internally — and Double.toString is specified to return the shortest decimal representation that round-trips back to the same double. So when you start from a literal like 1.005, valueOf gives you the decimal 1.005 rather than the binary-exact value, which is what most callers actually wanted.

A reasonable rule of thumb, based on which entry point one is using:

• Use new BigDecimal("...") for literals and external decimal inputs that arrive as strings
• Use BigDecimal.valueOf(double) only when you already have a double in hand and need the best possible decimal view of it (still risky after computations, but better than new BigDecimal(double))
• Never use new BigDecimal(double) unless you explicitly want the exact binary-to-decimal conversion, which is rarely what callers expect

It is worth one further caution. If the double you are wrapping has come out of arithmetic — rather than out of a literal — then the value you are now “viewing” through valueOf may already be quite far from the decimal you think you started with. valueOf is an honest decimal view of whatever double it receives; it is not a cleansing ritual that retroactively repairs precision lost earlier in the calculation.

Why HALF_UP can be the wrong default

HALF_UP is, in fairness, intuitive. It matches what most of us were taught with a pencil and paper in school, and it has the agreeable property of making the answer easy to predict in any individual case. The trouble is that, when applied repeatedly to many values, it can introduce a systematic drift, because every tie is resolved in the same direction.

It is worth pausing for a moment on a small irony of the IEEE 754 standard here. IEEE 754’s default rounding mode for binary floating-point arithmetic is round to nearest, ties to even — which is, in effect, HALF_EVEN. Each individual double operation in Java is specified to behave as if its result had been rounded that way. The mild irony, then, is that when developers reach for BigDecimal precisely because they are now worried about decimal correctness, many of them then choose HALF_UP, thereby reintroducing the very bias that IEEE 754 had originally been designed to avoid.

Bias demo: HALF_UP vs HALF_EVEN

Suppose, in order to make this concrete, that you have a population of values which all happen to land exactly on ties at two decimal places. (This kind of thing happens more often than people tend to expect, particularly after divisions and intermediate scaling steps.) The behaviour of the two main candidate rounding modes, side by side, looks like this:

• HALF_UP: 1.005 -> 1.01, 1.015 -> 1.02, 1.025 -> 1.03 … always nudging up
• HALF_EVEN: 1.005 -> 1.00, 1.015 -> 1.02, 1.025 -> 1.02 … nudging toward whichever side has an even last digit, and so alternating which side “wins” on ties

The structural difference is that HALF_EVEN alternates which direction it pushes ties, while HALF_UP always pushes them the same way. This is the underlying reason why banking and accounting systems often prefer HALF_EVEN: across a large enough population of values, the alternation cancels out into something close to zero net bias, while HALF_UP accumulates a small but consistent upward drift that becomes visible at scale.

The bias in action

A short program makes the difference clearer than any description does:

import java.math.BigDecimal;
import java.math.RoundingMode;

public class BiasDemonstration {
  public static void main(String[] args) {
    String[] ties = {"1.005", "1.015", "1.025", "1.035", "1.045",
                     "1.055", "1.065", "1.075", "1.085", "1.095"};
    
    BigDecimal sumHalfUp = BigDecimal.ZERO;
    BigDecimal sumHalfEven = BigDecimal.ZERO;
    BigDecimal trueSum = BigDecimal.ZERO;
    
    System.out.println("Value   -> HALF_UP / HALF_EVEN");
    for (String tie : ties) {
      BigDecimal bd = new BigDecimal(tie);
      BigDecimal roundedUp = bd.setScale(2, RoundingMode.HALF_UP);
      BigDecimal roundedEven = bd.setScale(2, RoundingMode.HALF_EVEN);
      
      System.out.println(tie + " -> " + roundedUp + " / " + roundedEven);
      
      sumHalfUp = sumHalfUp.add(roundedUp);
      sumHalfEven = sumHalfEven.add(roundedEven);
      trueSum = trueSum.add(bd);
    }
    
    System.out.println();
    System.out.println("HALF_UP sum:   " + sumHalfUp);
    System.out.println("HALF_EVEN sum: " + sumHalfEven);
    System.out.println("True sum:      " + trueSum.setScale(2));
  }
}