What shapes are you looking for? I would assume just the basic shapes, circle, square, triangle, right? 3D shapes or just 2D?

Each of those shapes has a different number of lines. Your algorithm can do what is called **edge detection**, which is a very simple and well known algorithm to implement. Edge detection finds the outside lines of the shape, then you can count the number of lines. A circle has 1 line, a triangle has 3, a square has 4, etc.

For more complicated shapes, your algorithm can then calculate the angle between each line. Obtuse triangle, right triangle, etc.

For a rectangle, see if 2 lines consist of more pixels than the other two lines.