I'll guess the simplest architecture that gets you quickly into the territory you want to play in would be a bunch of cheap, dumb slaves that can move and do stuff, but can't do much thinking, and a computer that does nothing itself, but tells the slaves what to do.
One advantage of the swarm notion is that you're already committed to two-way wireless communication. You might as well take advantage of this to offload the thinking from the swarm bots to a computer with a compiler, lots of RAM, etc, so you can start work in an environment you already know how to use. You can emulate the separation between swarm intelligences on a single computer, get algorithms that you like, and then, when you know what computing resources are required, re-implement with distributed intelligence.
The alternative--starting with smart swarm bots--might be what you want in the end, but I think you don't want to start there, because you'd have to do all your trial development on a computationally crippled platform, and you'd have to choose the platform before you really know what capabilities are required.
Since you're not strong on the building-robots side, I'd start with mass-market, utterly stupid "bots", like cheap remote controlled toy cars, and get your computer to emulate the kid driving them (i.e., interface your computer to the hand-held controller). This won't get you any data from swarm bot to computer, but it's an inexpensive way to start moving things. Eventually, you'll want a microcontroller on each bot, to run sensors and send data to the computer. Maybe the BASIC stamp-based kit you linked would be good for this, but you certainly don't want to do AI development on a stamp. You'd spend all your time fighting limitations, and none on algorithm development. I'll guess things like the Basic stamp are better for simple, one-off robots, and not as cost effective for a swarm--even a swarm of three or four--as a microcontroller to which you download object code, such as an AVR.