Community Render farm. Want to be apart of it?

Started by Asthenia, April 01, 2013, 12:10:23 pm

Previous topic - Next topic

Are you interested in participating in a community render farm?

Yes!
4 (100%)
No!
0 (0%)

Total Members Voted: 4

Asthenia

Hey guys! I'm thinking about creating a "community render farm". Basically you will  need to download Squidnet RenderFarm Manager and Hamachi and we can all use each others systems as a collective to render our massive render files! Is this something you guys are interested in doing? Please let me know if you are! There might be a small fee(like 5 dollars) because I may have to purchase hard drives.

Tangled-Universe

I like these kind of ideas, as TG is one of those typical pieces of software which can never have enough horsepower :)


I'm only very concerned about how such a community renderfarm would work and be managed.


Logically contributors are also allowed to render a piece one would presume.
On top of that contributors could also expect priority if they "deliver" more rendertime compared to another user.
So some kind of queuing system.


Another practical concern is how it is controlled what's being rendered.
I can already see projects being submitted with either ridiculous amounts of frames or insane resolutions or worse of all: a combination of those with certain rendersettings causing unnecessary overhead, waiting time and possible crashes of certain rendernodes.
On the rendering side there's probably no end to what aspects you need to take into consideration if you build a farm based on many different machines with very different available CPU time.

How and what do you think about these aspects so far?
I'm curious to hear your plan.


Cheers,
Martin

Asthenia

Well first off I will be managing it. With the software I can give some machines higher priority on the system. You can split up the huge images into up to up to 625 smaller images and each little piece can be rendered by multiple systems. The software supports unlimited amount of systems and control. I could set it up so I can only submit jobs meaning people would have to send me there TG files and then I do the rendering. If it came down to that I could set this whole system up as a subscription you pay each month. The only reason I would do that is if it wouldn't work if everyone submitted there jobs on their own.

Also at first when we don't have as many systems I can control it so people can't submit 500 frame jobs  but at a later time when we have plenty of systems we can do large jobs 

cyphyr

April 01, 2013, 01:39:10 pm #3 Last Edit: April 01, 2013, 04:29:51 pm by cyphyr
Hmm it's a great idea but I do feel there a number of issues that make it quite unrealistic.

Firstly I agree with all the points TU has mentioned.

Also I would be uncomfortable in letting someone remotely control my computers and even if I could be convinced this was ok I'm not too sure there would be that much spare computer cycles available. If my pc's are switched on then I'm using them, they don't get left running with nothing to do.

Further more running a computer costs money (probably different amounts in different countries) but you want to charge (albeit a small amount) for the service. I think that will cause some "bad feeling"; I don't mind paying for a service that I'm not contributing to but if I'm already donating my computer resources it seems a little rich to ask for money as well.

Lightwave used to have a distributed render client "StealthNet". It was dropped. Huge security issues basically.

If Terragen had been originally written like Seti@home or Folding@home (google if you don't know them) then it "could" have a better chance of working since once data has been distributed then the individual calculations are VERY small. I don't know if this was ever possible. Also rember that many Terragen projects use many hundreds of MB of data files (objects and textures etc). GI prepass files can run into many GB of data. All this takes time to load, populations take time to calculate (often hours) as dose GI prepass data and all the while no actual rendering is being done. Should a system be switched off or the owner of the system decide to use it for their own work then any loading or calculating of data will be lost and still no rendering has been done.

I would hope that at some point in the future some sort of "cloud" rendering option for Terragen could be implemented. There was talk of leveraging "Amazon cloud computing" a while back but I think it ran into the same problems TU and I have mentioned.

There are also licensing issues regarding content.

I know this sounds very negative but I would LOVE for something like this to happen. I just think it will require a lot more thought and work to implement.

Good luck

Richard

Ps I think you may want to try some different software, Hamachi is only good for connecting two computers via the internet (and apparently has file sharing issues)  and
Squidnet RenderFarm Manager (the Deluxe is the free version) only supports 8 cores, that's just one computer in most cases.
www.richardfraser.co.uk
https://www.facebook.com/RichardFraserVFX/
/|\

Ryzen 9 3900X @3.79Ghz, 64Gb (TG4 benchmark 6:20)
i7 5930K @3.5Ghz, 32Gb (TG4 benchmark 13.44)

Asthenia

I understand your concern. You made many very very vaild points that I never thought of! My only concern security wise is Hamachi.


The reason I talk about charging a small fee is because I WILL need to get a few TB's of space to be able to hold files as people send them to the NAS(which would be on my end). Basically you will move your project file to the NAS and then you submit the job.


I could set limits on how much resources it will consume on each computer. If you are willing to contribute more power the help other people render then you in return will get more power for your renders.


Also, I have a full copy of Squidnet, so I have unlimited amount of cores/systems that can be in my farm.


Also another point. If some of you guys are interested in doing a small render community just to help each other out with CPU power that would be cool! I personally have a total of 26 cores to render with myself.

jaf

You may want to look at BOINC and then here: http://en.wikipedia.org/wiki/List_of_distributed_computing_projects


Projects like Burp and Renderfarm.fi use BOINC.  Community animation projects would seem to fit well...... not sure about non-animation or especially NDA work.
(29Oct20) Ryzen 1800x, 970 EVO 1TB M.2 SSD, Corsair Vengeance 64GB DDR4 3200 Mem,  EVGA GeForce GTX 1080 Ti FTW3 Graphics 457.09 (29Oct20), Win 10 Pro x64, Terragen Pro 4.5.43 Frontier, BenchMark 0:10:02

visualaspirant

Did anything like this ever get off the ground? Or are there any community based render farms that do Terragen similar to SheepIt for Blender?