I believe... correct me if I'm wrong... that you can't go very fast with omni-directional wheels.... right?
Eventually you'll hit a speed where the rollers on the omniwheel are spinning fast enough that they do not have enough grip to grab the ground, causing the robot to slide. Typically this is a high speed though. If you compared this to typical wheels, however, then yes, omniwheels do have a lower top speed. This is usually not problematic, however.
The significant "problem" with omniwheels is that since the wheels are designed to allow the wheel to slide in any direction save the axis that it is being turned on without power, that it can also be easily PUSHED side to side.