Four years ago, astronomers had a spectacular realization for a new type of science they could perform: they could, at long last, search for the presence (or absence) of a cosmic optical background. We have a theory about where light comes from in the Universe:

  • we have the leftover light from the Big Bang, now shifted into microwave portions of the spectrum,
  • we have the light generated from stars, thought to be distributed inside galaxies, galaxy groups and clusters, as well as a few stars and star clusters in the intracluster medium,
  • we have the light reflected off of neutral clouds of matter,
  • and we have the light generated by heated matter, which generates a cosmic infrared background.

In theory, the only “cosmic optical background” that ought to be there is the light generated from stars, which should be confined to within galaxies and larger bound collections of matter, plus a little bit of additional reflected light from within those same structures. But from Earth and even from the space within our Solar System, we cannot conduct these measurements; there’s too much “stray light” from sunlight reflecting off of tiny particles in interplanetary space to find true darkness.

But there are five spacecraft that have traveled far enough away from Earth — Voyager 1 and 2, Pioneer 10 and 11, and New Horizons — that all of that stray light, known as zodiacal light, is finally faint enough that they can answer questions about the darkness of deep space. Although the first four don’t have the right tools to make those critical measurements, New Horizons does, and is still fully operational to boot. After a careful analysis by the New Horizons team, here’s what we’ve learned about the cosmic optical background.

JWST deep field vs hubble

A portion of a JWST deep-field image, shown with the Hubble observations as its counterpart. Within the JWST field are a significant number of objects not seen by Hubble, showcasing JWST’s ability to reveal what Hubble could not, thanks predominantly to its longer-wavelength capabilities. All of the light produced by stars, black holes, and excited atoms contributes to the cosmic optical background, but excess light beyond that would present a cosmic puzzle.

Credit: NASA, ESA, CSA, STScI, Christina Williams (NSF’s NOIRLab), Sandro Tacchella (Cambridge), Michael Maseda (UW-Madison); Processing: Joseph DePasquale (STScI); Animation: E. Siegel

Imagine that you were around at the start of the hot Big Bang, and you were able — from an omniscient point of view — to watch the history of the Universe unfold. Initially, you’d find that the Universe was completely illuminated. An incredibly dense and energetic bath of particles, antiparticles, and radiation, including photons, persisted everywhere and in all directions, with energies easily reaching and exceeding those of optical wavelengths. Over time, however, the Universe expands and cools, and the light within it sees its wavelength stretch, reducing its energy and transforming it from ultraviolet to optical to infrared and even longer, lower-wavelength energies. As the Universe cools by a large enough amount, neutral atoms form.

But then a new source of light arises. Once neutral atoms form, the matter in the Universe begins to gravitationally contract, drawing matter into the densest regions of all and causing streams of gas to intersect. As these gas clouds collapse and fragment, their innermost densities increase, trapping heat as they contract down farther and farther, until a critical threshold is crossed, and stars form. Those stars once again begin lighting up the Universe with optical (visible) light, and by the time we arrive at the present day, more than 13 billion years later, we expect that more than two sextillion stars have formed, distributed in galaxies, groups of galaxies, clusters of galaxies, and even grander structures.

large-scale structure simulation

The evolution of large-scale structure in the Universe, from an early, uniform state to the clustered Universe we know today. The type and abundance of dark matter would deliver a vastly different Universe if we altered what our Universe possesses. Note the fact that small-scale structure appears early on in all cases, while structure on larger scales does not arise until much later. If this is a good model of reality, the optical light we see should arise from the predicted astrophysical sources within this framework, and nothing more or in excess of that.

Credit: R. E. Angulo et al., MNRAS, 2008; Durham University

There are additional processes that occur in these collections of matter that should also give off visible (optical) light.

  • Black holes should accrete matter, heating up to such great temperatures that visible light is emitted.
  • Shining stars emit ultraviolet light, which — owing to cosmic expansion — gets shifted into the visible portion of the spectrum before it arrives at our eyes.
  • Starlight strikes neutral grains of matter, like dust, heating it up and causing the electrons within it to get excited, where they cascade down the energy levels and emit light as they do so.
  • And starlight strewn omnidirectionally throughout the Universe strikes clouds of matter, which reflect a portion of that light, creating additional sources of optical radiation.

However, it’s largely the stars within galaxies, and to a lesser extent, the black holes within those galaxies, that are expected to be primarily responsible for the optical light in the Universe. One of the great tests of this idea would be to launch a telescope far out of the Solar System, beyond where the grains of ice and dust present in interplanetary space would reflect sunlight, and measure the faint amount of cumulative light coming from all the external sources within the Universe. If the sum of all that light — i.e., the cosmic optical background — matches the light predicted to come from galaxies and their black holes, then this helps confirm our modern picture of the Universe. If not, however, we may find out that deep space isn’t completely dark, and that perhaps there’s more to the Universe than we presently imagine.

Nasa's deepest view of galaxies in the night sky using JWST.

This ultra-distant view of the Universe comes from a portion of the JADES survey, leveraging JWST’s capabilities. Although there are trillions upon trillions of stars producing the light powering these galaxies, they extend back for tens of billions of light-years in space. Many unresolved, faint galaxies exist in the space between these galaxies, contributing to the cosmic background of light. Quantifying that background, across all wavelengths, is a vital task to ensure our cosmological model accurately reflects reality.

Credit: JADES Collaboration

We’ve already taken an extraordinary census of faint galaxies at a wide variety of distances, using powerful deep-field techniques at high resolution with observatories like JWST and Hubble, and also using wide-field techniques at lower resolution with telescopes like the Sloan Digital Sky Survey, Gaia, and Euclid. But all of these telescopes suffer from a profound drawback: they’re not far enough away from the Sun. As a result, there’s the risk that scattered light, both from the Milky Way and also from the presence of zodiacal dust, is polluting these observatories. In order to probe the depths of interstellar space, we need to eliminate these sources of, for lack of a better phrase, “light pollution.”

You can imagine some clever ways to try to deal with this light. You can imagine taking observations when an observatory is in Earth’s shadow, but the zodiacal light is still present, and will still pollute your observations. You can imagine looking well out of the ecliptic plane, where the zodiacal light is faintest. But even if you do this, the background brightness of the sky is about a factor of 15 greater than the unresolved extragalactic light; Hubble and JWST both are swamped by this noise. The only realistic option is to go far, far away: to great distances, where the density of interplanetary dust particles — the ones that reflect sunlight and create the zodiacal light — is so small that it only contributes negligibly.

While the stars, galaxies, and Milky Way are familiar sights in the night sky, they are joined here by the faint zodiacal light that arises from light (mostly direct sunlight) reflecting off of Solar System dust particles. Profusely present in the inner Solar System, zodiacal dust is fundamentally limiting when we collect faint observations of the distant Universe.

Credit:
ESO/B. Tafreshi (twanight.org)

But at long last, we now have such a candidate observatory: the New Horizons spacecraft. Presently at a distance of approximately 57 Astronomical Units (where 1 AU is the Earth-Sun distance), it’s one of only five spacecraft ever launched by humanity that’s presently on a trajectory that will cause it to escape from the Solar System. It also contains the most remote active camera ever deployed, and at its location at such great distances from the Sun, is basically free of the zodiacal light foreground. While Hubble, in low-Earth orbit, is incapable of making high-quality measurements of the cosmic optical background, New Horizons, armed with its LORRI (Long-Range Reconnaissance Imager) instrument, is now in prime position to do exactly that.

The first attempt to conduct these measurements, and to quantify the cosmic optical background with New Horizons, was conducted back in a 2021 paper, and they reckoned with a number of issues. They modeled the effects of:

  • camera noise,
  • remnant scattered sunlight,
  • excess off-axis starlight,
  • reflections off of crystals that were emitted by the spacecraft’s thrust,
  • and instrumental effects,

so they could remove them from the observations. They ignored observations that were close to the plane of the Milky Way, and instead focused on observations that were as far out of the galactic plane as possible. They then quantified the light that they did observe, compared it with the expected overall contribution of stars and galaxies, and drew conclusions about just what, if anything, they could learn about any excesses present within the observed cosmic optical background.

Diagram depicting the solar system with labeled comets, asteroid belt, zodiacal cloud, and planets including Jupiter and Saturn. Inserts highlight the zodiacal cloud against the dark expanse of space, offering new horizons for exploration.

The dust between the planets that scatters sunlight our way is not from the asteroid belt (depicted here in green), but from periodically disrupting comets that spend much of their time near the orbit of Jupiter. The zodiacal dust dominates for distances in the Solar System interior to the orbit of Saturn. Beyond that, the dust density drops off precipitously, but zodiacal light remains important until the end of the Kuiper belt.

Credit: SWRI/SETI Institute (Andrew Blanchard, David Nesvorny and Peter Jenniskens)

That study, based on seven fields that were acquired back when New Horizons was closer to the Sun (42 to 45 AU away), they inferred a cosmic optical background intensity of 15.9 ± 4.2 nanowatts per square meter per steradian (where a steradian is a measure of solid angle on the sky; there are a total of 4π steradians across the entire sky), which was approximately double the amount of light expected from unresolved stars and galaxies. It seemed that the data was pointing to a surprise: perhaps there was more light out there in the Universe than the known and anticipated stars and galaxies could account for.

However, there were other sources of foreground light that could have potentially contaminated these observations. The diffuse galactic light was a severe contaminant, as stars within the Milky Way would emit light that would become scattered off of interstellar dust. Scattered starlight, where stars outside the LORRI instrument’s field-of-view would contribute starlight that still made its way into the camera. Based on a large suite of observations made with the New Horizons spacecraft, the New Horizons team now conducted new work here in 2024, work that’s been accepted for publication in the Astrophysical Journal, where they acquired new LORRI images that were explicitly designed to minimize these foreground contributions.

A spacecraft travels through the dark expanse of space, depicting a distant planet and its moon in the background, heralding new horizons.

From its journey to well beyond the orbit of Pluto, NASA’s New Horizons has taken many images of space, enabling it to measure the extragalactic optical background without the pollutive effects of the nearby zodiacal dust.

Credit: NASA/New Horizons

This new work contains a number of interesting finds that significantly refine what we had concluded previously. The diffuse galactic light contribution, in particular, was refined substantially, and was found to be much greater than the previous work had indicated. In the earlier analysis, the “anomalous” component of the cosmic optical background, which couldn’t be explained by foregrounds or by excess light from unresolved stars and galaxies, was quantified at 8.8 ± 4.9 nW/m²/sr, which was significant at the more than 3-sigma level: highly suggestive, but not entirely conclusive.

Now, however, the new analysis, driven by a reanalysis of the diffuse galactic light, more than halves both the amount of excess light attributed to the cosmic optical background, and also halves the significance of the result. As the new paper states very clearly:

“While our observations can accommodate a modest [cosmic optical background] anomaly relative to the amplitude of the IGL, we cannot falsify the simpler hypothesis that the [cosmic optical background] is due entirely to the known population of galaxies.”

How we arrived at this new conclusion is of the utmost importance, since the New Horizons data is basically the first spacecraft to ever measure the brightness (or faintness) of space far beyond the effects of zodiacal light pollution.

planck first all-sky map

The majority of the dust signatures seen in our galaxy arise from our galaxy itself, as this full-sky map from the Planck satellite shows. However, when it comes to the entire Universe beyond the Milky Way, it is unknown whether the source of the unidentified optical light is from unseen galaxies or some other, possibly dust-like source.

Credit: Planck Collaboration/ESA, HFI and LFI Consortium

The largest improvement is in the number of fields acquired with LORRI to conduct the analysis. Whereas the original 2021 paper only used 7 fields to estimate the cumulative optical background, this new paper leverages 16 fields for estimating the background, plus 8 fields for calibration purposes. This creates a lower-uncertainty result, as well as a more robustly calibrated result than the prior analysis.

Originally, the 2021 analysis discovered that there were elevated background levels of noise (that gradually decayed away) each time the LORRI instrument was powered on. (It’s off when not in action, to conserve power.) In order to accommodate this effect, they waited an estimated four minutes for the elevated background to decay away sufficiently. (Its ultimate, underlying cause is still unknown.) However, this new analysis shows that this elevated noise continues to decay away even after four minutes has elapsed, requiring a recalibration, which was included in the newer analysis. This type of improvement is essential in reducing your systematic errors.

However, there were multiple improvements made to contributions to foregrounds, as well as additional improvements made in estimating the contribution to the measured signal from diffuse galactic light, and these play a major role as well.

An image of a dark section of space featuring numerous stars and nebulous clouds, with a mix of bright blue stars and wispy, cloudy formations, offering a glimpse into new horizons.

When interstellar dust grains absorb light, they heat up and shine in the infrared regime. This produces optical signals as well when the dust scatters light from nearby stars, and is known as either galactic cirrus or the diffuse galactic light. Quantifying this correctly is an important problem in estimating the cosmic optical background from within our galaxy.

Credit: Marc-Antoine Miville-Deschenes/IRAP

For foregrounds, they discovered that the contribution from scattered galactic light is slightly greater than was estimated in the earlier analysis: 0.10 nW/m²/sr, as opposed to 0.07 nW/m²/sr previously. They also found the need to correct for the emission from the n=3 to n=2 energy level transition in hydrogen, where excited hydrogen atoms emit photons of a specific wavelength (656.3 nm) as they de-excite, which was not included in the earlier analysis. And finally, they found that the contribution of adjacent bright galaxies, even though they were excluded from the field, contributed to the foreground as well, in addition to the scattered starlight, integrated starlight, and integrated galaxy light that had been previously quantified.

Travel the Universe with astrophysicist Ethan Siegel. Subscribers will get the newsletter every Saturday. All aboard!

As far as the diffuse galactic light was concerned, a superior far-infrared calibrator was used (from the Planck satellite, replacing the IRIS data used earlier), and the actual LORRI field-of-view was used to estimate the aperture, rather than a simple circle of a fixed angular radius. But perhaps the largest correction arose from a superior self-calibration method using the 8 high galactic latitude (5 north, 3 south) fields acquired in this new analysis, instead of using an estimator from 2017, from the team which first suggested using New Horizons to measure the cosmic optical background.

This plot shows estimates of the cosmic optical background, with error bars, compared to the predicted estimates of the overall optical background expected from known sources (colored, shaded lines). With the revised calibrations included in this work, denoted NH24, the measured optical background is highly consistent with known astrophysical sources.

Credit: M. Postman et al., Astrophysical Journal accepted, 2024

When you put all of these effects together with the new measurements, the latest analysis finds that the estimated strength of the cosmic optical background is much smaller than previous estimates: about half of what the greatest estimates with New Horizons data had revealed and only about two-thirds of what the original (2021-era) New Horizons analysis, by the same team, had suggested. Instead of a result that was significant at the more than 3-sigma level, which only comes along with about a ~0.3% chance of being a fluke, this result is highly consistent with the expected background from the known sources of light: stars, galaxies, black holes, and excited gas.

The remaining anomaly is only of approximately 1.5-sigma significance, which comes along with approximately a 15% chance of being a fluke: a value that’s far too large for comfort. In addition, it calls into question previous assertions, as new estimates of the diffuse galactic light leads to a revised value for the report of an earlier, 2022 paper by the same team (the lower gray bar, as opposed to the upper one), suggesting that there’s scant evidence supporting a detection of excess sources of light beyond what’s predicted by known physics. As the authors themselves conclude:

“If our present [cosmic optical background] intensity is correct, however, it means that galaxy counts, [very-high-energy] γ-ray extinction, and direct optical band measures of the [cosmic optical background] intensity have finally converged at an interesting level of precision… the simplest hypothesis appears to provide the best explanation of what we see: the cosmic optical background is the light from all the galaxies within our horizon.”