The shortcomings of AI
AI failures are probabilistic and context-sensitive: performance emerges from the interaction among model, prompt, and task. Deterministic expectations therefore produce brittle safeguards.
Suppose an AI evolves something analogous to sight. The unsettling question is not merely whether it can see, but whether greater capability would make it any less likely to kill people.
R1Capability is not predictable conduct
Normal software is expected to follow explicit instructions reproducibly. AI behaves differently: the model supplies learned tendencies, the prompt steers which tendencies surface, and the task context changes what counts as a good answer. A system can therefore succeed repeatedly and still fail unexpectedly when one part of that combination shifts. More capability does not automatically supply the goals, judgment, or restraint needed to use that capability safely.
R1Evolution is a question, not a guarantee
The evolutionary comparison exposes the problem, but it does not prove that AI will evolve like humans, develop perception in the same way, or become destructive. Those are questions raised by the material, not established outcomes.
R1Test the interaction, not just the model
Before relying on an AI workflow, rerun the same task with altered wording and edge-case contexts. Record where the answer changes, then place human review or a hard constraint at those unstable points—the practical logic of Human-AI Complementarity.