Man, I remember how much this concept threw me when I was starting out in electronics many decades ago! Bottom line, it's just a shortcut. Rather than draw all those lines back to the supply (in a circuit diagram), just use a little symbol to represent all those connections. A schematic can get pretty cluttered by a bunch of lines running back to the other side of the supply (unless it's a simple circuit, then, often the lines are drawn back to the supply)! But, how do you show the complete circuit path without drawing all those lines? Use a symbol to represent those connections.
Now, to confuse things a bit more, there is "positive ground" and "negative ground". All that means is "which side of the battery/power supply" are we going to call the "reference" (I know you don't like that word, but that _is_ what it is). Reference just means "refer back to" and is simply the point where all voltages are referenced from. So, if I call the negative side of the battery "the reference", then that voltage (the voltage on the negative terminal) becomes zero and all other voltages are "referenced" to (or measured from) the negative side of the battery. THAT is a "negative ground" (or "negative reference" or "negative earth" or "negative ground reference" or "negative ground system", etc, --I know! Why so many terms for the same thing?!?)
A "positive ground" circuit is simply a circuit diagram where the positive side of the supply was chosen as the reference point. These days, this kind of diagram is rare. All of the logic families still in common use (that I know of) are negative referenced, and I have almost never seen an analog circuit that wasn't referenced to the negative rail (I do remember seeing an analog circuit that was predominately composed of PNP transistors that had a positive ground).
In case this still makes no sense, try this. ALL voltages are measured _across_ something. It makes no sense to say that the positive side of an AA battery is at 1.53 volts, unless it is understood that the voltage measurement was made _across_ the battery. In other words, the "negative" side of the battery was arbitrarily selected as the "zero potential point".
OK, now I sound like I'm contradicting myself, right? How can the "negative" side of the battery be at 0 volts if I just said that a voltage has to be across something? Here's how: take the negative probe (usually black) and place it on the negative terminal of the AA cell. Now, place the positive probe (usually red) on to the negative terminal of the AA cell -- yes, you heard me right, put both probes on the negative side of the cell. What voltage do you get? Zero volts, right.
If you're screaming, "Are you nuts?!?" at this point, then I'm screaming back, "Generally, yes... but in this case, No!!!" When you put your negative probe on the negative side of the battery, you were selecting that as your reference point. Then, when you place the positive probe of your meter on the negative terminal of the cell, you were "measuring the voltage there". You were measuring the voltage across the negative terminal. If you keep the negative probe on the negative terminal of the cell, and move the positive probe to the positive terminal, you are measuring the voltage across the cell--or in abbreviated terms you are measuring the voltage at the positive terminal (with the silent agreement that the negative terminal is the reference point).
Now, how did the "reference point" come to be called "ground"? As David Adams pointed out, there are 3 types of ground: Earth, Signal and Chassis. In the beginning there was just Earth ground. Because, in our daily lives, we stand on the Earth, it made sense to make everything we touch be at the same potential as the Earth. That way, there will be zero voltage (or nearly so) between our feet and the things we touch, thus saving us from electrocution. Because chassis' are one of the things attached to ground potential, the chassis just started being called "ground", even if it wasn't actually attached to the Earth. It's like how we came to say "Kleenex" when referring to facial tissue, even if the brand of the facial tissue wasn't Kleenex.
This trend extended to all circuits, even ones that weren't even connected to the mains system (i.e. battery powered systems), or even to a chassis. Back in the days of tubes and the high voltages needed to drive them, chassis' were made of metal and became the conductor that completed the circuit back to the power source. Because of the high voltages involved in tube circuits (on the order of hundreds of volts), it became a standard safety measure to connect the chassis to earth ground via a third wire that literally was connected to the Earth. So, the term "ground" became synonymous with "common" and "reference". BTW, the chassis of a car is used as the "ground return" to the battery--usually to the negative side of the battery, but there are rare cases of the chassis connected to the positive side.
Are you getting that this is all arbitrary? It all stems from three things:
1. A voltage is a measure of a potential _difference_ and thus is ALWAYS across something: a battery, a resistor, a Base-Emitter junction, an array of components, the sky and the earth, the Sun and Mars, ...
2. Drawing lines back to the supply is a drag (and potentially very messy).
3. Agreement.
BTW: sometimes BOTH (or ALL, in cases of multiple supplies) sides of the power supply are represented by labels, e.g. VCC, VDD, VSS, V+, V-, GND... It's all about keeping the schematic clean and readable.
BTW2: Why is the non-positive terminal called "negative" if it's really 0 volts? Is it? What if you measure the negative terminal from the positive terminal? In other words, what if you put your black probe on the positive terminal and measure the negative terminal with your red probe? You get a negative voltage, right? The reason they label battery terminals “Positive” and “Negative” is because until you _use_ the thing, there is no way of knowing which terminal is going to be chosen as the reference!
