The weirdest problem in physics | Sean Carroll

Despite celebrating its 100th anniversary, quantum mechanics remains one of the most successful yet least understood frameworks in science. We use it to build semiconductors and lasers, and its equations predict experimental results with staggering precision. Yet, at its heart, a deep mystery remains regarding what the theory actually tells us about the nature of reality. From the collapse of the wave function to the elusive unification with gravity, physics is currently suspended between a theory that works perfectly and a fundamental understanding that remains out of reach.

Physicist and philosopher Sean Carroll argues that while we have mastered the "Core Theory" of the particles and forces that make up our everyday lives, the path forward requires us to confront the weirdest problems in physics—specifically the measurement problem and the quantization of gravity.

Key Takeaways

• The world is made of fields, not particles: What we perceive as particles are actually vibrations in quantum fields; "fermions" make up matter because they cannot pile on top of one another, while "bosons" carry forces.
• Measurement breaks the rules: Quantum mechanics is the only theory in history where the act of observation plays a fundamental, yet undefined, role in the physical laws.
• The "Core Theory" is complete for everyday life: The Standard Model combined with General Relativity explains everything humans experience directly, from biology to chemistry, though it fails to explain the Big Bang or black holes.
• Success is a problem: The Standard Model currently fits experimental data too well, leaving physicists without the anomalies and "wrong" results needed to discover new physics.
• Emergence defines complexity: Understanding fundamental physics is crucial for defining boundaries (what is impossible), but it is largely useless for predicting complex, emergent systems like biology or sociology.

The Shift from Particles to Quantum Fields

To understand the current state of physics, one must look back to the late 19th century. Physicists of that era believed they were on the verge of a complete theory of nature, viewing the universe as a dual harmony of matter (made of particles) and forces (emanating from fields). However, this classical view crumbled when applied to the atom.

Under classical electrodynamics, an electron orbiting a nucleus should emit light, lose energy, and spiral inward, causing all matter to collapse in a fraction of a second. The fact that atoms are stable forced the development of quantum mechanics. Today, we know that the distinction between particles and fields was an illusion. In modern physics, everything is a field.

Fermions and Bosons

If the universe is entirely composed of fields, why does matter feel solid? The answer lies in the two categories of quantum fields:

• Bosons (Force Carriers): These fields can pile vibrations on top of each other. Like a guitar string vibrating with increasing intensity, boson fields can occupy the same state, allowing forces like magnetism and gravity to macroscopic scales.
• Fermions (Matter Particles): These fields, which include electrons and quarks, obey the Pauli Exclusion Principle. They cannot occupy the same space in the same state. This resistance to overlapping gives matter its structure and solidity.

The resolution to this early mystery of quantum mechanics... is number one, they're waves. That's what they are. They're waves in a field.

The Measurement Problem: Physics’ Unfinished Business

While the mathematics of quantum mechanics—specifically the Schrödinger equation—describes the evolution of wave functions perfectly, it leads to a philosophical dead end known as the measurement problem. A wave function describes the probability of finding a particle in a certain state. Left alone, this wave evolves deterministically. However, when a measurement occurs, the wave function appears to "collapse" into a single definite outcome.

This introduces a unique element into physics: the observer. In classical mechanics, measuring the position of a planet does not alter its orbit. In quantum mechanics, measurement fundamentally changes the system. The theory provides rules for what happens when a measurement occurs, but it fails to define what constitutes a "measurement." Does it require a conscious human? A camera? A interactions with air molecules?

The Role of Consciousness

Because the definition of an observer is vague, some have speculated that consciousness itself causes the collapse of the wave function. While this is a tempting philosophical solution, most physicists, including Carroll, reject it. It is more likely that consciousness is an emergent phenomenon resulting from physical laws, rather than a fundamental driver of them. Nevertheless, the lack of consensus on the measurement problem indicates that our understanding of quantum foundations is incomplete.

The Core Theory and the Standard Model

Despite these foundational mysteries, physicists have constructed the "Standard Model" of particle physics. This framework describes the building blocks of the observable universe:

• Quarks: Up and down quarks constitute protons and neutrons.
• Leptons: Electrons and neutrinos.
• Forces: Electromagnetism, the weak nuclear force, and the strong nuclear force.
• The Higgs Boson: The field that gives other particles their mass.

When combined with Einstein’s General Relativity (which describes gravity), we get what Carroll calls the Core Theory. This theory is remarkably successful. It accounts for every experiment ever conducted on Earth. It explains the stability of matter, the fusion in stars, and the chemistry of life.

The Power and Limits of Emergence

The Core Theory highlights the concept of emergence. Just as one does not need to know the position of every atom in the Earth to calculate its orbit around the Sun, one does not need to understand quarks to do biology. Different layers of reality—physics, chemistry, biology, sociology—operate with their own autonomous rules, even though they are ultimately derived from the layer below.

However, fundamental physics still sets the boundaries of possibility. For example, claims of telekinesis or perpetual motion machines can be dismissed not because we have investigated every specific claim, but because they violate the established laws of the Core Theory regarding forces and energy conservation.

The Crisis of Success and the Future of Physics

Paradoxically, the biggest problem facing fundamental physics today is its success. In the past, progress was driven by experimental anomalies—data that contradicted current theories. Today, the Core Theory fits the data so well that the Large Hadron Collider and other experiments have failed to find deviations that would point toward new physics.

We know the Core Theory is not the "Theory of Everything" because it cannot account for:

1. Dark Matter: We see its gravitational effects in the cosmos, but it does not fit into the Standard Model.
2. Quantum Gravity: We understand gravity in weak regimes (apples falling), but the math breaks down in strong regimes, such as the center of a black hole or the Big Bang.

If you want to win the Nobel Prize in physics, you don't show that Einstein was right. You show that Einstein was wrong.

String Theory and the Search for Unification

String theory emerged in the 1980s as a promising candidate to unify gravity with quantum mechanics. By positing that particles are tiny loops of string, the theory naturally produces gravity and leaves room for all known particles. However, string theory has proven incredibly difficult to connect with experimental data.

The scientific community finds itself in a strange interim period. We have a theory that works too well for everyday phenomena but fails to explain the extreme origins of the universe. To move forward, physics needs new data that breaks the current models—unexpected results that force scientists back to the drawing board.

Conclusion

Physics is often viewed as a collection of cold facts, but it is ultimately a human endeavor driven by curiosity and the confrontation of the unknown. We have achieved a rigorous understanding of the matter that makes up our bodies and our planet. Yet, the deep questions—how quantum mechanics reconciles with gravity, what constitutes dark matter, and what truly happens when we measure a quantum system—remain open. The next great leap in human knowledge will come not from proving what we know, but from discovering exactly where our current understanding fails.