CHAT GPT 4.0 plugins

Started by xpez2000, July 16, 2023, 10:19:32 PM

Previous topic - Next topic

xpez2000




Has anyone considered creating a plugin for Terragen using CHat GPT 4.0?

I have no idea how to do it but apparently this is supposed to be not a huge issue with the latest Chat GPT since it can crank out code for many things.

Productivity tools like an geo-mesh auto tiler /batch render  and FBX importer that links textures so you dont have to do it manually would be awesome -If I had to imagine some new plugins to create. 

Would something like this be ok to do or is it not possible???

Kevin Kipper

Hi xpez2000,

You could create productivity tools like you've described using Python or another programming language. 

You could also use chatGPT as tool to help you understand and write certain parts of that code within the programing language of your choice.  For example, I recently delved into the principles behind the math involved in a 4 x 4 matrix, an approach which Terragen uses when saving population caches in popxml format. ChatGPT help me understand the nuances involved and was like having an instructor available to answer my questions as they arose. 

On my Windows desktop I have a couple of batch (.bat) files for batch rendering Terragen projects overnight.  They are basically text files, so there's no reason that a productivity tool couldn't be written in Python, for example, to generate the batch files themselves. 

Terragen 4 Professional now includes a Remote Procedure Call (RPC) feature which allows third party programs to communicate with the open Terragen project.  A geo-mesh auto tiler application could probably be coded to do this, that is load and manipulate the position of said geo-mesh.  I'm not sure about an FBX importer, but more to the point, you could definitely write a program that looks at the MTL file associated with an OBJ formatted object and write out a Terragen clip file with the image textures assigned to the proper inputs of a Default shader.  The clip file could then be loaded into the internal node network of the OBJ within Terragen.  I've done this sort of thing to "transfer" surface material settings from Daz 3D assets in OBJ format to Terragen, which can be a big timesaver if an OBJ has a lot of surfaces.

Be sure to take a look at the recent blogs we've posted about Terragen RPC here:
https://planetside.co.uk/blog/hello-rpc-part-1-scripting-for-terragen/
https://planetside.co.uk/blog/hello-rpc-part-2-error-handling/


xpez2000

Thanks for the feedback. I want to experiment and try to build something. 8)

digitalguru

Someone at work mentioned using ChatGPT to write code, so I tried it, the results are patchy at best. For apps like Maya/Python it can do a reasonable job so long as you give it small chunks of code to do, the overall logic of the script should be up to you.

It's only as good as the data it can pull in from its database, and since there's a lot of info on the net for the above mentioned programs it can do a good job. However, I'm just starting to learn Houdini, and here it is absolutley appalling - it "hallucinates" answers, making up functions that don't exist in the app. When you post its suggestions don't work, it will then post a "debug" script that includes even more bogus code. As there's not much documention on Terragen RPC out there, I think it would be just as inaccurate.

To use it successfully, break your intended app down line by line, testing each function as you go - and check elsewhere to confirm its answers.

I've written a script Terragen for Maya (https://grahamcristie.gumroad.com/l/wxcbx) which does some of the things you mentioned:
- Can setup a tiling camera to split a terrain into tile to export to .obj
- Imports geometry from Maya and can convert Maya and Arnold shader to the Terragen default shader and links textures

Script currently does this with clip files - I might get around to updating this with the RPC one day so it does this directly. Question to Kevin - has the RPC been updated? I used it a couple of years ago.

If you were interested in using Terraman drop me a PM if you're using Maya later than 2022 as I'd need to compile against later versions.

Matt

In early 2024 I found that GPT4 did a pretty good job of coding with Terragen RPC if I simply pasted a copy of the RPC doc first. I didn't make any mistakes with the RPC calls as far as I remember. I was working with AutoGen and the OpenAI API rather than the ChatGPT interface, but I would expect ChatGPT to perform similarly well.

Newer models such as Claude might be even better.

RPC hasn't had any recent updates.
Just because milk is white doesn't mean that clouds are made of milk.

digitalguru

Interesting and good to know! Did you have to paste your RPC code every time you had a question?

I'm diving into a project using Terragen again, so might take the opportunity to replace sections of my script with direct RPC calls.

Matt

For APIs like Houdini's that are not very well represented in LLM training but also not contained in a single document to copy and paste into a conversation, you might need to try something different.

Tools like Cursor, Aider and maybe even Copilot (I have not tried yet) should let you provide them with references. Another option is AutoGen. It can execute code, see errors, make corrections, rinse and repeat. Combined with chain-of-thought and other techniques, LLMs are getting ridiculously good at this stuff.
Just because milk is white doesn't mean that clouds are made of milk.

Matt

I was using AutoGen. Yes, I did have to paste the code back in after a while as it would start to drift. This may have been a limitation of the context length of GPT4, and I wouldn't recommend that workflow for a large codebase.

I don't have a workflow for AI assisted coding on my large codebases yet.

Inspired by AutoGen, and the relative simplicity of agent-based workflows, I actually spent a couple of weeks writing my own AI agent-based coding assistant. I could write a document describing an assigment, and a path to a folder of code, and then have a conversation with the system to help it generate the code for the assignment. Then I came to my senses and got back to working on Terragen!
Just because milk is white doesn't mean that clouds are made of milk.

Matt

If you mean did I have to paste the RPC docs in every time, I don't think I did, but that may have been because most of the difficulties it had were not with the RPC-related code if I remember correctly.
Just because milk is white doesn't mean that clouds are made of milk.

digitalguru

Thanks for those suggestions. As mentioned, I'm new to Houdini, and learning Python in that app is something I want to tackle. However code reference and tutorials are bit scattered, particularly when you're starting out and just want to do something like "create a Solaris material and attach some image files". AI assisted app are good for asking these questions (depending on the app - not so great for Houdini it seems) - but I'll have a look at those other suggestions!

Kevin Kipper

I'm creating and using Python scripts on a daily basis for use with Terragen projects.  I've found they can automate repetitive tasks, and enhance user productivity. 

As Matt mentioned RPC hasn't been updated, but we are keeping track of new features that we plan to incorporate in future updates of RPC.

I use ChatGPT all the time.  Not to "write the code for me", but rather to help me learn to write code better.  Often I'll ask ChatGPT how to write a line or block of code and find that its approach was completely different from what I was thinking.  Usually, the insights it provides are better than the approach I had in mind...however, sometimes it doesn't get the scope of what I've asked and then its reply is just wrong.

I don't copy/paste the Terragen RPC docs into ChatGPT, mostly because I'm not asking it to generate an entire script from the docs and my idea.

Recently, I tried creating a Blackmagic Fusion comp from a Python inspired ChatGPT script in order to load image files rendered from Terragen.  Unfortunately, it's not working yet, and part of that is my lack of understanding the Fusion comp format and that ChatGPT doesn't know it either.

Anyways, the combination of Terragen RPC, Python and ChatGPT is a really good thing, and if more users would start tinkering with it and sharing their results and scripts I think it would greatly improve the overall creative and artistic experience of using Terragen.

digitalguru

Quote from: Kevin Kipper on September 23, 2024, 02:26:30 PMOften I'll ask ChatGPT how to write a line or block of code and find that its approach was completely different from what I was thinking.  Usually, the insights it provides are better than the approach I had in mind.
Yes, that's the best approach and I've found works quite well writing for Maya/Python. Haven't tried it yet with RPC, but will do soon. It works well when I give it a few lines of code and it will suggest list comprehension to reduce it to a line.

As mentioned it doens't work well with Python in Houdini, and hallucinates a lot, which is frustating as that's the code I'm eager to learn.