Turn Internet Servers Into Free Supercomputer

[voyager2]

Hi All,

I had another one of my "ideas":
I was thinking of a better way to let the average Joe have his own super computer, than hooking up two thousand PlayStation 3s

How about using internet servers?
Join the servers, somehow, i've heard that Server to Server transfer is really fast, then have them share the load.

You can sign up for free server space at 000webhost , with 1.5GB of space, and 99.9% uptime.

Anyone tried to do this?
It might be my next project...

Voyager

[madsci1016]

You can sign up for free server space at 000webhost , with 1.5GB of space, and 99.9% uptime.

Nope, won't work. All hosting companies use shared servers. They can host thousands of accounts on one server. So you would have to sign up for thousands of accounts before you had the power of a single server.

You can get a 'dedicated host', which means you are the only account on a server, but they cost hundreds of $ per month to have. By the time you get a cluster going, might as well have bought all those PS3s.

[voyager2]

The thing is, with this, you don't have to pay for the electricity, and you don't realy need to have a server as such, the server computer will still be computing it all.

[madsci1016]

The thing is, with this, you don't have to pay for the electricity, and you don't realy need to have a server as such, the server computer will still be computing it all.

Your missing the point, the server's are already operating at near 100% load just serving the pages for thousands of websites per computer. There's no room left for any 'computing'. If there were, they would add more websites to the server till it was at full load to increase profits.

[chelmi]

The thing is, with this, you don't have to pay for the electricity, and you don't realy need to have a server as such, the server computer will still be computing it all.

Your missing the point, the server's are already operating at near 100% load just serving the pages for thousands of websites per computer. There's no room left for any 'computing'. If there were, they would add more websites to the server till it was at full load to increase profits.

Exact. Moreover, most of these servers won't allow you to run your own program.

[Soeren]

Nah, it's hard enough to be allowed running a CRON.
Bin that idea and look into Beowulff clusters if you really need that much power (and when you gain some more electronics knowledge, you will be able to make your utility meter run slow, or even backwards - not that I'd recommend it though).

[rbtying]

You'd be better off building your own server farm than using a hosting provider - even the more expensive VPS systems aren't really better than your computer until you're paying $100s per month, at which point you can afford the server farm.  Plus most providers will kick you out for using up all the CPU =). 

[TrickyNekro]

Apart from the things the guys above said....

Man, we are at the age of dodeca-core architectures.... Peace....
If you need that much power get an Opteron...


Still a six core Phenom for a home PC is an over kill....

[voyager2]

Apart from the things the guys above said....

Man, we are at the age of dodeca-core architectures.... Peace....
If you need that much power get an Opteron...


Still a six core Phenom for a home PC is an over kill....

What do you mean Phenom? I'd go Phenom II!
You have no idea what I'd do with a computer like that!
I'd overclock it to 7.127GHz and have it compute an infinite loop in 6 seconds...
If the Cray Supercomputer can do it, so can a Phenom II.

[madsci1016]

The six cores are Phenom II's.

[TrickyNekro]

Apart from the things the guys above said....

Man, we are at the age of dodeca-core architectures.... Peace....
If you need that much power get an Opteron...


Still a six core Phenom for a home PC is an over kill....

What do you mean Phenom? I'd go Phenom II!
You have no idea what I'd do with a computer like that!
I'd overclock it to 7.127GHz and have it compute an infinite loop in 6 seconds...
If the Cray Supercomputer can do it, so can a Phenom II.

Phenom is the word.... (Family guy)

BTW
I'm really thinking of phenom II.... It's a time for upgrade.... But I desperately lack the money....

Would you get the 6 core, or the 4 core just for the higher freq....
With my time at Phenom overclocking exp. I found that freq DOES matter heavily....

But six core sounds so sweet.... :-PPP

[SeagullOne]

What?! 6-core processors?!
Do those exist? And AMD Processors too? Doesn't the intel i7 still only have four cores?

Wow...the exponential growth of processing power is going up faster than I thought...

Let me know if I'm confused about the i7 though.

[chelmi]

What?! 6-core processors?!
Do those exist? And AMD Processors too? Doesn't the intel i7 still only have four cores?

Wow...the exponential growth of processing power is going up faster than I thought...

Let me know if I'm confused about the i7 though.

http://en.wikipedia.org/wiki/List_of_Intel_Core_i7_microprocessors#.22Gulftown.22_.2832_nm.29

The Gulftown has 6 cores.

[madchimp]

I believe they have 12 core opterons

[dunk]

the issue with using any multi processor system is writing software to take advantage of multiple processors.
on most multi core desktop systems only one of the cores ever approaches full CPU load.
by their nature can servers make far better use of multiple CPUs, particularly if they are performing multiple concurrent tasks.

if your processors are on different physical machines the problem gets *much* harder. the CPUs now no longer share the same IO and storage system so making them interact quickly enough to be useful is very challenging.
much of the research in this area has been aimed at the server market, allowing server farms to use different servers to handle requests without end users knowing they are receiving results from different physical machines.

as for building your own networked multi-computer cluster out of virtual machines,
it's kind of pointless from a performance point of view but it is highly educational.
for experimenting, rather than using "free server space" with all the restrictions that entails i would use multiple virtual machines on a computer at home.

if it is performance you are looking for without having to do the hard work of setting it all up yourself,
have a look at some of the offerings from Google or Amazon.
these companies let users run code on their datacenters.
the advantage here is you get an API that makes it appear as if you are using a single host but storage, CPU time, network capacity, etc, are distributed between as many physical machines as is required.
http://code.google.com/appengine/docs/whatisgoogleappengine.html
or
http://aws.amazon.com/


dunk.

[amando96]

Damn... someone hates lag   

[arachne]

How about using the power of our internet-connected PCs when they r idle. Something like SETI project

[voyager2]

I'll look into that dunk, sounds good...

How about using the power of our internet-connected PCs when they r idle. Something like SETI project

You read my mind.
People might not like some weird app uploading goodness knows what to a server
Never the less I might look into it... 

[TrickyNekro]

Basically, the most easy to do thing, is a grid like configuration...

Provided you know how to code in that particular language these PC's use...

You can "chop" your "many many repeats" program, in sub programs and give a separate tank
to separate computers on the GRID...

So if a single computer needs one hour to do 1000 math... 10 computers would need 6 minutes to do 100 math...
And so have your answer, simulation, or what ever you need, in one tenth of the time need by a single core...

[chelmi]

You can "chop" your "many many repeats" program, in sub programs and give a separate tank
to separate computers on the GRID...

So if a single computer needs one hour to do 1000 math... 10 computers would need 6 minutes to do 100 math...
And so have your answer, simulation, or what ever you need, in one tenth of the time need by a single core...

With a big limitation though. This is the theoretical speedup, achievable only if there is no dependencies between your tasks.
In reality, you will almost never be able to parallelize your application in completely independent sub-tasks. At some point during
their execution they will need to synchronize and talk to each other. And you will waste time here.

So it's just to say that parallel computing does not come down as putting computing resources together. It also implies connecting them smartly
and most importantly (in my opinion), writing an application in such a way that you will minimize dependencies between tasks.

Conclusion is, it all depends on what you want to do (type of application, quantity of data...). What is it in your case voyager2?

[Razor Concepts]

Folding at home is a good example of it working, but of course it is a very specialized task and is not very time sensitive (you don't need one part to finish the rest, so no computers are stuck waiting)

[voyager2]

You can "chop" your "many many repeats" program, in sub programs and give a separate tank
to separate computers on the GRID...

So if a single computer needs one hour to do 1000 math... 10 computers would need 6 minutes to do 100 math...
And so have your answer, simulation, or what ever you need, in one tenth of the time need by a single core...

With a big limitation though. This is the theoretical speedup, achievable only if there is no dependencies between your tasks.
In reality, you will almost never be able to parallelize your application in completely independent sub-tasks. At some point during
their execution they will need to synchronize and talk to each other. And you will waste time here.

So it's just to say that parallel computing does not come down as putting computing resources together. It also implies connecting them smartly
and most importantly (in my opinion), writing an application in such a way that you will minimize dependencies between tasks.

Conclusion is, it all depends on what you want to do (type of application, quantity of data...). What is it in your case voyager2?

My evil plan is to compute pi, to at least 5000 places...
Hence the little message in my signature
And of course there's the satisfaction of telling people that I own a super computer...hehe
My pi algorithm is very simple (1 step!):

Code: [Select]

360 / theta * sin(theta / 2) = piThe smaller theta is the more accurate pi is.
Unfortunately for me, although my computer (you don't want to know the specs!) has the capacity of computing this, my programming skills do not  
I should also mention that sine MUST be in degrees NOT radians...

Also, since the computation is only done once, I wouldn't want to spend lots of money on PS3 clusters, renting a supercomputer, etc...

Before you ask, why can't I do this without a computer, let me tell you this:
You try calculating the sin of half 0.0000000000001 
 

[msprague]

Pi to 1,000,000 decimal places.
http://newton.ex.ac.uk/research/qsystems/collabs/pi/pi6.txt

[chelmi]

My evil plan is to compute pi, to at least 5000 places...
Hence the little message in my signature
And of course there's the satisfaction of telling people that I own a super computer...hehe
My pi algorithm is very simple (1 step!):

Code: [Select]

360 / theta * sin(theta / 2) = piThe smaller theta is the more accurate pi is.
Unfortunately for me, although my computer (you don't want to know the specs!) has the capacity of computing this, my programming skills do not  
I should also mention that sine MUST be in degrees NOT radians...

Also, since the computation is only done once, I wouldn't want to spend lots of money on PS3 clusters, renting a supercomputer, etc...

Before you ask, why can't I do this without a computer, let me tell you this:
You try calculating the sin of half 0.0000000000001 
 

It seems to me that this is not the kind of problem that will take advantage of parallel computing. Ask yourself the question "how can I split the problem is small subtasks?"

For instance you can split the problem in two parts:
- 360 / theta
- sin(theta / 2)

the final multiplication need the two parts and cannot be parallelized.

You have 2 sub-tasks that can be computed in parallel.  Maximum speedup: < 2

You just need 2 computers! You can add billions of computers to your cluster, this result won't change.

To really take advantage of distributed computing, you need an application with a lot more of data entry.
The Folding@Home and SETI project are good examples. PCs are used to perform simulations or to process data from radio signal.
The amount of data processed is HUGE.

One last comment on your project. My guess is the difficulty is not access to computing power, but the coding.

[voyager2]

Pi to 1,000,000 decimal places.
http://newton.ex.ac.uk/research/qsystems/collabs/pi/pi6.txt

Pi has been calculated too 5,000,000,000,000 places.

My evil plan is to compute pi, to at least 5000 places...
Hence the little message in my signature
And of course there's the satisfaction of telling people that I own a super computer...hehe
My pi algorithm is very simple (1 step!):

Code: [Select]

360 / theta * sin(theta / 2) = piThe smaller theta is the more accurate pi is.
Unfortunately for me, although my computer (you don't want to know the specs!) has the capacity of computing this, my programming skills do not 
I should also mention that sine MUST be in degrees NOT radians...

Also, since the computation is only done once, I wouldn't want to spend lots of money on PS3 clusters, renting a supercomputer, etc...

Before you ask, why can't I do this without a computer, let me tell you this:
You try calculating the sin of half 0.0000000000001 
 

It seems to me that this is not the kind of problem that will take advantage of parallel computing. Ask yourself the question "how can I split the problem is small subtasks?"

For instance you can split the problem in two parts:
- 360 / theta
- sin(theta / 2)

the final multiplication need the two parts and cannot be parallelized.

You have 2 sub-tasks that can be computed in parallel.  Maximum speedup: < 2

You just need 2 computers! You can add billions of computers to your cluster, this result won't change.

To really take advantage of distributed computing, you need an application with a lot more of data entry.
The Folding@Home and SETI project are good examples. PCs are used to perform simulations or to process data from radio signal.
The amount of data processed is HUGE.

One last comment on your project. My guess is the difficulty is not access to computing power, but the coding.

Indeed the coding is the hardest part, but its the funnest 
I tried to create a JavaScript that could do it all on my website...
But what luck! JavaScript doesn't have a simple sine command...

There is another algorithm that could take advantage of computer clusters...
And of course I can't remember it
Much more complex, 5-6 steps, but I can't remember exactly right now...
I'll post again when I find it...

[voyager2]

Here's that algorithm:
It starts off with the values:
A=1, B=0.5, C=square root of 2 / 2, D=0.25

Code: [Select]

Let E = A
Let B = 2 * B
Let A = (A + C) / 2
Let C = square root of C * square root of E
Let D = D - [B * (A - E)^2]
Then pi = (A + C)^2 / (4 x D)
Loop
In six loops it can calculate pi to 170 digits.
In 20 loops it can calculate pi to 1 million digits...