High resolution terrain textures (test)
-
Ever since I made this thread I have a "Machines" folder on my desktop to remind me that I should give this texture upscaling another shot. Few weeks ago I even managed to install ESRGAN like M123 suggested. I used this tutorial to install it properly, if anyone is interested:
http://alphagarg.blogspot.com/2019/01/esrgan-neural-network-ai-for-upscaling.html
Now I finally gave up on this idea, I think it is not worth it at current state. 256x256 is very limiting to make much of a difference and let's be honest, there is too much work and who would even care in the end.
I asked the maker of dgVoodoo about the possibility of injecting higher resolution textures using his wrapper and he said it might be possible some day in the future, so maybe then it would make more sense to work on this...
-
I have been messing around with a new model for ESGRAN. I have been upscaling some textures for Dungeon Siege with it, with great results for ground and wood textures, so I wanted to give it a try with Machines.
First I tried the multiplayer desert level and results were very impressive. I used a "debanding" model to fix color banding in textures, then applied a "misc" model that works great for ground textures:
https://i.imgur.com/ySarrG2.png
This produces a 1024x1024 texture and unfortunately since the original version has a 256x256 texture resolution limit, and the 1.5 version also has a limit of 512x512, I can't even use it in the actual map and need to downsize it which defeats the whole purpose. In my opinion the limit should be further increased to at least 1024x1024 in the next update.
Then this new "misc" model failed where I had the highest hopes. The "Finger Of God" level has a mixture of rock with ground and grass textures. If you look the top brown and green part of the texture you can see what used to be a pixelated mess, now resembles ground with grass, if you use a little imagination. But the problem is the lower part where there should be mountain rocks the algorithm also creates what resembles blue grass:
-
@antrad What if the problematic ones are corrected using Photoshop? Because the second one is impressive.
-
@Pisarz Well, an artist can do whatever he wants, but I am not one, I can't fix things manually. My only hope is finding a better upscaling model, or one that at least recognizes the rocks, so I can mix the two textures; take the ground from this model and copy the rocks from the other.
All this is really a waiting game now, results are very promising, but someone still needs to make good upscaling algorithms and models that recognize different types of terrain. It would be a waste to go through all the trouble fixing things manually, just for a better solution to pop up after a few months that does it automagically. I will try more tests with different types of terrain.
-
@antrad I published your other post on the Discord group just to make sure it reaches @martin.
About your last post, I was thinking about a situation where there are few exceptions given a good overall performance. My ignorance about that technology is almost absolute, but given that the results are promising, maybe some textures could be fixed manually (not necessarily by you, but by me):
The one you posted
Green (moss)
Brown (Autumn)
-
@Pisarz I think we have different opinion what is not good about that texture. For me personally there should be nothing that resembles grass on those rocks. I marked some of it in red.
https://i.imgur.com/Q9OkAkx.png
The whole blue-ish area for me just doesn't look like rocks. For comparison here is one texture upscaled from Dungeon Siege. Here the algorithm worked well and it looks like a surface of a rock:
https://i.imgur.com/y85sBK8.png
At 128x128 resolution there is very little info for the algorithm to recognize stuff. The reason desert texture worked very well is because it is 256x256 and algorithm has enough information to recognize things more clearly and enhance them.
-
This is a bit off-topic, but I want to show you what I did with Dungeon Siege using ESRGAN. I have made a comparison video, please watch in fullscreen and at 1080p to see the difference more clearly. I think the results are amazing, though I must admit there are many things that give very poor results, so I only worked on things that gave good results.
https://www.youtube.com/watch?v=HL8F0Tdh_YY
I also must show you this example in the link below. Look at the original texture in top-left corner and look at the result. It is mindblowing ! You can see individual leaves of grass, there are larger leaves around the tree stump, there are like yellow squished flowers. I just couldn't believe it when I saw it, it was just a pixelated mess.
https://i.imgur.com/2enh2Rj.png
This one is actually not used in this video, it is upscaled twice, so it would take too much time to do for every texture and my computer wouldn't be able to handle it.
If you are are interested you can check out my blog where I wrote more about the whole process:
https://antonior-software.blogspot.com/ -
@antrad I'm planning my own approach of using GAN's and CNN's in up-scalling process, not sure when though. This requires an small research in dataset preparation, deconvolution patterns elimination techniques, recent publications and some extra processing powers.
There should be implementations available, which are utilizing OpenCL instead of CUDA, not sure if you encounter a problems with running out of GPU's memory or how long it would take to optimize new model/dataset parameters on a single card (2-12 hours perhaps).
-
@antrad You are right! I think that that texture does not look like rocks only: rather than a pure surface, I visualized roots, dirt, dead moss, etc. (like the image below).
Something similar like in the game Hunt: Showdown, where there are huge piles of dead organic material, such as the base of a tree long fallen, with all its roots exposed still clenching dry soil and little rocks (can't look for an example right now).
That would be a design choice only if we were to define how a satisfactory output should look like, but your example provides what should become marks on the rocks, and your video shows clear improvements which are almost magical (like putting on my glasses!).
I think that a good way to see how to improve this specific texture would be to take a look at its adjacent ones during gameplay, but to give you a better response, I should learn more about this technology.
The one with the yellow flowers is truly impressive!
Also, making that surface look like more than just rocks could change the game's visual languaje, which is intended to be bit cartoony, so that kind of upgrade could go against how the whole composition should look, which reminds me of games' concept art: the results in the video are so good that it looks as if the game became more "itself", as if it where turned into an even more "original" version, closer to concept art, rather than merely being upgraded.
So it would be interesting to see the result you are looking for and compare the scene where it is being applied, just like you did in the video.
I currently don't have time, but it would also be interesting if I modify with Photoshop this texture to make it look like what I think you want.
I'll see your blog's content in more detail after the exams; it looks very interesting! And I loved this title:
Fixing a GPU by baking it in an oven - It actually works !
-
@martin I am doing all this upscaling on my CPU, because I don't have a Nvidia GPU, so it is a very slow process. I do run out of memory if I try to upscale anything larger, I am limited to upscaling just 256x256 textures and smaller. Also I can not train my own models, even though I have some interesting ideas I would try to do. If you do end up doing some upscaling yourself, have a look at my topic about removing dithering and color banding, which should be applied BEFORE you do any actual upscaling.
@Pisarz The reason I think that part of texture should be just rock is because in The Finger Of God level it covers the mountain and cliffs, and that dark green "line" around is like a border between ground with vegetation and mountain rocks. As for retaining cartoony look, I think the machines and buildings are colorful, but not much cartoony, and terrain looks very realistic. The multiplyer desert map is probably the best looking map and you can see it looks very realistic, probably the most realistic it could look using technology from 1999. I mentioned some new textures are 1024x1024 in size, so I can not use them in the game, because original game has a 256x256 texture limit and Martin's version has a 512x512 limit.
-
@antrad Now I see. I thought it was for something like a hill only.
-
@antrad said:
I do run out of memory if I try to upscale anything larger, I am limited to upscaling just 256x256 textures and smaller.
You might want to have a look at Image Enhancing Utility (formerly "crop-unscale-merge").
It can split images into tiles, upscale them with ESRGAN, then merge them together into a single image.
With an appropriate tile size, it should be possible to upscale large images without running out of memory. -
@M123 Thanks, but I saw that and in description it says "Split images into tiles before processing them with ESRGAN and merge them afterwards with as little visible seams as possible". That is not good enough, if someone else with a bit more RAM can make it perfect :)
-
@antrad The tiles are created with a configurable amount of overlap to help prevent artifacts around the edges. The overlapping areas are smoothly blended together - the differences between a single-piece and tiled upscale are unlikely to be noticed during gameplay.
I upscaled sand_b.bmp using the "Misc" model - once as a single piece, and once using 128x128 tiles, 32px overlap:
https://i.imgur.com/e9dYYU9.png
https://i.imgur.com/MRjyFfd.png
Another upscaling model or texture might produce a more visible difference in one-to-one comparisons, but hard seams don't appear to be an issue.
-
@M123 Well, that example looks perfect. Now if only it wouldn't take me 13-14 minutes to upscale one 512x512 texture :(
-
Allow me to be off topic once again and show you my finished Dungeon Siege texture project. It took more than 4 months and 3929 textures have been updated. The texture pack is 5.97 GB !
