1) USB or network attached
The simple option is a USB drive, plugged into one computer. You can unplug it and plug into another computer and even carry it around if you want to share, for example, videos.
Probably you want just a normal USB drive, but in case you want to get fancy…
A network attached drive plugs into ethernet on your wireless router, and is available to all the computers on your network.
Many routers have a USB socket, so you can use regular USB drive on the network.
2) You probably won’t come near to filling 4GB but if it’s in your price range, future-proof yourself. 4GB can be had below $100 US on sale. I see 8GB on sale now for $185.
Photos don’t take up much room (compared to music) and video takes A TON.
3) Get USB 3.0
They might all be USB 3.0 now, but just in case, get that. It can copy files back and forth much faster than 2.0 (where your computers have USB 3.0).
4) Portable drives are available. If your main goal is carrying stuff around, look for those. As Jose says, $60 gets you 2GB.
5) Look for sales at Amazon, Best Buy, etc.
Here’s a site that lists deals on all kinds of stuff (pretty much anything Amazon sells), and external HDs often show up. Kinja Deals