That computer SHOULD be fine. We're running it on a netbook and it doesn't have that much memory or processor speed. We are using ubuntu, though, mainly because ROS is stable and tested for it, not debian.
I'm running openCV and point cloud library with ROS and it's great, although it is MUCH faster to run it on our desktop PC when using point cloud library -- the Microsoft Kinect can produce 300Mb/s of data! That's a bit much for a slower computer :-P