Small galaxies are key tools for understanding structure formation and galaxy evolution. Traditionally defined as galaxies below a mass threshold of ~10^9 Msun, they have long been used to study the individual components of galaxies (stars, gas, chemical elements) and as archeological records of the conditions in the early Universe. Now we are finding gas-rich, star-forming galaxies in the 10^5-10^7 Msun mass regime. The physical properties of these extremely low-mass systems lie in the regime where theoretical predictions diverge and allow us to explore numerous questions about galaxy formation, survival, and evolution. In this talk, I will show results from a new frontier in low-mass galaxy research and discuss the discovery and characterization of gas-rich systems hovering at the brink of what we call a galaxy.