32-bit hardware can physically support 32-bit word instructions and addresses. In 32-bit hardware, this limitation exists because 32 pins are used as instruction inputs into the CPU. For this reason, 32-bit computers can only address up to 232 bits of memory (4GB) and they can only crunch up to 32-bits of data at a time. So, 4GB is
the maximum amount of memory that can be used by a 32-bit system.
64-bit hardware can physically support 64-bit word instructions and addresses. That means every clock tick on a 64-bit machine can process twice as many bits of data at a time compared to a 32-bit machine. It also means that longer instructions and addresses can be used. 64-bit computers can technically address up to 264 bits of memory (2EB). Needless to say, that’s a lot of memory. In fact, it’s so much addressable memory that hardware manufactures physically limit the amount of memory that can be installed in 64-bit computers to 8GB or 16GB, etc. so that they don’t have to account for all that addressable memory.
One of the biggest performance bottlenecks that modern computers have is the hard disk. Most hard disks (except for SSD) are mechanical, so there is a significant lag between when a request from the disk is issued and when it is returned. These mechanical hard disks use a special cache to reduce that cost, but at the end of the day the platters still have to rotate and the head still has to seek. Compared with the speed of system memory, hard disks don’t have a chance.
64-bit makes it possible to have more system memory installed. The more system memory installed, the less often data has to be fetched from the disk and the faster the computer will perform, provided that computer has a 64-bit operating system. Since the operating system handles the interaction between disk and memory, if the operating system is 32-bit but the underlying hardware is 64-bit capable then the computer will still only be able to utilize 32-bits of memory.
Oddly enough, the additional processing power of 64-bit computing will likely not be realized by most people. The reason for this is partly because the available software applications just don't have a need for it. But it is mostly because like 32-bit CPUs, 64-bit CPUs only process one instruction at a time. And 64-bit CPU instructions are not all that different from 32-bit CPU instructions (for good reason). The biggest performance gain with 64-bit will undoubtedly come from additional system memory.
So, are the advantages of 64-bit systems worth upgrading your computer for? Maybe. If you need the performance boost that 64 bits of addressable memory can offer or if you need to run 64-bit software then by all means, go to 64-bit. But if you don't fall into either category then maybe you can save a little money in the short term and stick with a 32-bit system.