New New Terrain System

[SolarPortal]

Hi,

Just wanted to show the progress of the new terrain system. Due to the recent terrain system from ogre not doing what we need, we have decided to write a terrain system from scratch unique for skyline.

We have already got the ability to load a heightmap and split the terrain into multiple chunks and generate multiple lods for each stage.

Here is a pic of the terrain using 8192 x8192 vertices wide over a 16km distance:

Clipboard Image.jpg (271.32 KiB) Viewed 44159 times

lol.... its a bit heavy atm and uses 9.6gb of ram so this pic is from a 64bit app. Its currently displaying the lowest lod values of 4x4 cell grid for each terrain chunk and in the demo when you get closer, each chunk evntually becomes 128x128 wide by stepping through all the lod stages of 4,8,16,32,64,128. We are building a system that scenes and terrain made in Gen1 will be portable into this terrain but we want the ability to expand and have massive worlds... lol.. saying it like 16km by 16km with a vertex per 4 meters is not large enough lmao

Because of the way we have built the terrain, we should be able to edit chunks of the terrain pretty quickly like before.. Just a material system to build where the aim is to get at least editable textures.

[lordalmighty1]

this is good to hear, a unique terrain not ogre might be nicer in the long run?!.

can I make a request/suggestion if possible?.

a random terrain generation system option possibly might be nice to go with this?!!.
not that I don't like making them from scratch but a usable system for random gen terrains might be something others and me aswell would use?. something like a list of sliders for amount of water, mountains, hills/plains etc
might be hard to do I'm no coder like but I'm sure there be some sort of algarthym (cant spell that apparently lol) for it somewhere online maybe?


here is a blog on random terrains, means nothing to me LOL.. but maybe helps you 2 with ideas possibly if you wanted to
http://blog.habrador.com/2013/02/how-to ... rrain.html

[SolarPortal]

it certainly is a doable task on this new terrain system, but it will take some time; the adding the image to the terrain and generating the data is done but creating the images is a bit more work lol Nice idea though and it would be cool to have this feature.

Current Terrain status now... normals for lighting and tangents for normal maps are working:

Clipboard Image.jpg (117.45 KiB) Viewed 44150 times

Whats left since the mesh side is almost complete for now:

• Lightmap - for terrain shadows.
• Material for terrain shading. Currently running the pbr material
• Skirt edges or stitching for the tiles(you can see the cracks as it goes to lower lod)
• Editing & Splatmaps. Hoping that the editing tools we have fit easily into this new terrain

[TattieBoJangle]

Looking great that 9.6gb tho it will be some job reducing that but I'm sure you are up to the task. When you get round to the tool side would be nice to have a few more terrain textures slots to paint with

Also to lower that meme usage you could use a fixed array and have it shift every time the player moves this would help with the textures as well, also with that 9.6gb are the unused chunks being stored to the hard drive.

[SolarPortal]

When you get round to the tool side would be nice to have a few more terrain textures slots to paint with

oh, i think everyone using skyline is after this one including us... We think we can pull at least 8 textures off with the system and at worst, we can just make each terrain tile use a different set of textures to achieve this.

yes, the terrain array is a little larger for editing purposes otherwise we wont be able to know what vertices belong to what cell while the brush is editing lol End game will have less ram usage as certain arrays can be removed.

a 2048 sqrd rez terrain over "n" worldsize uses around 587mb with lods going all the way down a single square from 128x128.
Ideally, we shouldn't need to go below 8 or 4 squares across on the lowest lod. we think the current memory is still better than Gen1 terrains and can be optimized further. I reckon i can knock 200mb of the 500-600mb mark even for a 2048 res before streaming lol

The 9.6gb was a test to see what rez(8192x8192) we could get on a 16km world as the last terrain could only use a max size of 4096 even on 64bit. Theoretically with this terrain, if you have the ram, you can potentially go up to 16384+ but that would be an insane amount of ram. But streaming from file will be used eventually and should save a lot of ram. Having quick read/write to file is a must though so those with games running on an SSD will stream from disk much quicker.

Ultimately It all sits down to what is a good distance for rendering and what should the player see or how far. Using good level design to limit the view of the player and keep their interest is a must if high rez close ups are required. This doesn't help if your on top of a mountain though.

The idea is to get a good rendering distance on the terrain and have more terrain that is not yet visible. Couple that will loading and unloading tiles & lods plus streaming the data from a file on disk as we go will make the terrain far more efficient and larger with better resolution.

Our terrain is still early days and has only had 4 days work in it in between other bits, so with time it can be a kickass system designed to make what we all want out of it.

[epsilonion]

This is great to hear, a unique system means that you have first hand knowledge of the expectations and how far you cadd features in the long run as you know the system yourself instead of learning a newer one..

the 16km is amazing news, this with terrain paging wow just imagine..

While your at it you might as well kill 2 birds with one stome and implement the cutting holes in it for caves etc, and put in place some hooks etc for when your ready for the terrain paging system..

Now I must admit skyline has practically been redeveloped over the past year and all for the better, I must also admit the amount of crashes I am getting is less that half of Gen 1.

Heading in the right direction for definite, not only that we also respect the amount of work that has gone into the new build of Skyline.

Keep up the good work guys.... Can not wait till the system is intergrated into gen 2 and see the ow-some power of Skyline..

[SolarPortal]

wow, less than half the crashes of gen1... Good to hear!

Yeah, the Paging / Streaming is going to need some serious work to be a smooth workflow lol
As for the hook-in points for the paging, i guess you mean for your own unloading of entities etc... which we plan for skyline to do automatically as best it can for vegetation, and also we have a couple of plans for entities and zoning.

Thanks for the motivation booster

[SolarPortal]

Been a little while since i posted anything because of getting a machine working, but i have now got the Skirts working on the terrain which hide all those pesky holes between lods.

Here's a couple pics:

Clipboard Image (1).jpg (83.51 KiB) Viewed 44101 times

Clipboard Image (2).jpg (272.85 KiB) Viewed 44101 times

Clipboard Image (3).jpg (204.67 KiB) Viewed 44101 times

Next stage, is editing the surface of the terrain and then building a custom material system to have lots of textures or as many as a shader stage on DX11 and OpenGL will handle lol

[SolarPortal]

Next update for the Gen2 Terrain system.

The terrain from the basic app shown before is now integrated into skyline and working with the physics also.
Currently there is no Editing or Painting and the height data is loaded from a heightmap image.

Check the video out:

Note: There is a slight muckup in the video where i didnt set the far clip, so keep watching

This is a 20kmx20km terrain running at 130fps on this aging machine that contains a GTX 650TI for gfx.

Its getting there

[TattieBoJangle]

Amazing guys it sure is getting there and its HUGE good thing you where in a car

Great work!

[lordalmighty1]

cool cant wait for it to be finished

[SolarPortal]

Thanks This terrain was the same size as Just Cause 3.

[SolarPortal]

Awesome Newz! Raycasting to the terrain is now complete! Next Step >>> Edit Vertices.

[epsilonion]

Bloody hell your on a roll, considering its your own system its coming on leaps and bounds.
I am loving the 20km x 20km size, just think of the potential and what you could do with all that space... imagine making a motorway for the full length of it, shit it would take ages

The 120 fps is cool, just think what it will be like with occlusion culling and on a more up to date machine..

The video looks well cool (only viewed it on my mobile so far) but never the less it still looks awesome, can not wait for it to land now..

I know its a while off but with all the vegetation, paths, tools to change the terrain, paging/streaming and cutting holes in it. Once its all done it will be a great system and one to be proud, where you can say "I created that"..

Keep up the good work..

[SolarPortal]

Thanks liam
It should be quite cool once its complete!

[lordalmighty1]

so ray casting is done and vertices are next ok cool.. I have no idea what that all means yet but fair enough keep it coming lol

[SolarPortal]

lol No worries, i will try to explain them for you

Raycasting is a technique heavily used in games. A raycast is a line that is drawn from one point to another which can intersect 1 or more objects. For example, when you click an entity in the scene; we send a ray from the camera's position in World space in the direction we are facing to see if an object collision has occurred and return the distance to that object. If it succeeds then we use code that selects the object to display properties and act as if you just clicked it. The fps fires rays to detect if you hit a zombie to then trigger the health damage on the zomie.

In terrain sense, this is so we can select a point on the terrain for editing. Again, use the cameras position in world space and the direction we are facing to send a ray out from the cameras direction, if we hit terrain, then we sample where that ray has hit and return the world space coordinates.

Heres are couple of examples to showcase this:
First up is the terrain example:


Second is the FPS Raycast example:




For editing, we need the position to know what vertices exist at that location.

A Vertex is a single point in 3D space that is used to create a face from. A poly or triangle which you will have heard about doing your maya stuff is made from Vertices. A triangle uses 3 vertices at different positions to create a triangle, whereas a poly is made of 4 vertices to make a square.

Heres an example image of what a vertex is:

[Shando]

Nice explanation SP

The only other thing I would have mentioned would have been Normals - https://en.wikipedia.org/wiki/Normal_(geometry) ( as that's the bit I always seem to be fighting with ) and why they are so important to 3D Graphics!!!

[SolarPortal]

Normals & Tangents are needed in order to calculate the lighting angle on a flat face to make something appear lit rather than the same lighting all over the model. No normals or tangents, means no light direction at all. Without normals, it will look like an unlit model as the side of an object in ambient or shade would not know which direction the light is coming from to show light or dark. This is known in shaders as the TBN Matrix.

Normals are not normal maps, but normal maps require normals/tangents which is why you have several variations of blue in a normal map texture which indicates at a texture level what the normal at that point on the texture is which is why you see a bumpy surface in the engine. So Normals indicate whether the object is lit or in shade and the normal maps add surface normal variation to the face to alter the lighting to create the illusion of a bumpy surface.

Normals can be calculated simply by creating a line perpendicular to the face that it is being generated for. This is done by averaging several vertex positions to get the normals angle.

Here's a sample pic of a normal:


Now i mentioned Tangents in there also; these are similar to normals but work at adjacent angles of say 45 degrees to add extra information that can be read by shaders to do more. Binormals are a cross product of the normal and tangent which fills the TBN Matrix(Tangent, Binornal, Normal)

I could probably explain them better with more time, but i think you will get the gist of it.

One of the Gpu Gems books contain this information in more detail.

[epsilonion]

I like the sound of frustum.. it sounds quite sexual lol... "giv it some go on lad"

[SolarPortal]

lmao

[SolarPortal]

New information about terrain coming soon....

[Olander]

Nice work on the terrain! Well done.
Also excellent descriptions on the vertex/poly/triangle and normals.

A Terrain Feature Request?
There are a lot of heightmaps floating about. There is a system that 'Stamps' heightmaps on top of each other in Unity (named Gaia). This is very well done and is visually excellent yet easy enough to use.

Feature Request #1
The basis is that a heightmap is slotted then adjusted up/down and rotated with a visual indicator of what the result will be. Then simply click a button and the terrain is formed to what was visually there.

Feature Request #2
Export the newly made terrain as a heightmap and terrain texture map (splat map)...to share with other Skyline Devs. PNG would be great but even a Skyline map format would be acceptable.

Again great work on the terrain.

[SolarPortal]

Thanks Olander

Feature Request #1
The basis is that a heightmap is slotted then adjusted up/down and rotated with a visual indicator of what the result will be. Then simply click a button and the terrain is formed to what was visually there.

Nice idea and will try to remember this one for the future. The first thing we are aiming at is replicating what we had in Gen1 and then develop and expand on Gen2 terrain as it is infinately expandable now.

Feature Request #2
Export the newly made terrain as a heightmap and terrain texture map (splat map)...to share with other Skyline Devs. PNG would be great but even a Skyline map format would be acceptable.

This task will be added as Gen1 already had the ability to save a heightmap and each blend layer as an image that could then be used to recreate a terrain.
As for passing to skyline users, all you will need to do is pass the saved out terrain file, as this will contain the heightdata, blend map information and the textures that were used to paint the terrain much like Gen1's *.ter format.

Thanks and i should have more information by tomorrow

[Olander]

This task will be added as Gen1 already had the ability to save a heightmap and each blend layer as an image that could then be used to recreate a terrain.
As for passing to skyline users, all you will need to do is pass the saved out terrain file, as this will contain the heightdata, blend map information and the textures that were used to paint the terrain much like Gen1's *.ter format.

Thank you for the tip. Appreciated. Still learning and have not really gotten too much into Gen1 Terrain full capabilities...particularly the export portion.