anything related to UI in general, splash screens, UI widgets, input and output etc.

World Position of GUI Item

World Position of GUI Item

Postby Shando » 16 Sep 2017, 05:51

Hi All,

This may have been asked/answered before, but I couldn't find anything, so .....

If I want to implement Drag n Drop from a GUI button, how can I convert the button's position to a World Space position?

For example, this is what I would like to be able to do:

    Create a GUI with some buttons down the Left Hand Side. Each button has an image of an item that can be placed in the game.
    If I then click on a button and hold the mouse button down, I can drag the actual item (model) to a specific place in the game world.
    When I let go of the mouse button, the item will drop onto the nearest surface in the game world.
I suppose I'm really looking for something that is as simple as a

Code: Select all
sky.screenToWorld ( mouseX, mouseY )

function (which, unfortunately, doesn't look like it's simple to code!).

As an example, I found this on StackOverflow http://stackoverflow.com/questions/31613832/converting-screen-2d-to-world-3d-coordinates:

Code: Select all
mat = worldMatrix * inverse ( ProjectionMatrix )
dir = transpose ( mat ) * <x_screen, y_screen, 0.5, 1>

dir /= mat[3] + mat[7] + mat[11] + mat[15]
dir -= camera.position


The WorldMatrix would be calculated from the Camera as follows:

Code: Select all
worldMatrix = Translate ( x, y, z ) * RotateZ ( z_angle ) * RotateY ( y_angle ) * RotateX ( x_angle );


The ProjectionMatrix is somewhat harder:

The projection matrix can be calculated from the aspect ratio, field of view angle, and near and far planes.


So, this is starting to look like it might be too hard to do in Lua and I have also come across posts that seem to suggest I also need a ModelViewMatrix??

Unfortunately, I can't use raycasts as I can't guarantee that there will be an object directly under the cursor, and my tests indicate that raycasts give a result of 0, 0, 0 if no object is found.

Any help would be greatly appreciated.

Thanks

Shando
Ryzen 7 4800H 16GB GTX1650 Win 11 64
Love, Hope, Strength http://www.lovehopestrength.co.uk
User avatar
Shando
Skyline Moderator
Skyline Moderator
 
Posts: 560
Joined: 06 Mar 2013, 22:35
Location: Moffat Beach, Queensland
Skill: Programmer
Skill: Scripter
Skill: Level Designer

Re: World Position of GUI Item

Postby SolarPortal » 16 Sep 2017, 11:03

Hi Shando,

Currently, i have not experimented with drag and drop in the GUI editor, so i am unsure whether it is possible yet :oops:

However, we already have the worldpos to screen pos and commands as shown below:
Code: Select all
sx, sy = screen.worldPosToScreenPos( x, y, z );
sx, sy = screen.worldPosToAbsScreenPos( x, y, z )
state = screen.worldPosInFrustum( x, y, z )   -- State is 1 for true and 0 for false   
state = screen.worldPosInFront( x, y, z )   -- State is 1 for true and 0 for false   


However, it seems we are lacking the screen pos to world pos. so this will need to be added :)
Skyline Game Engine - Lead Dev.
Please provide as much info as possible when asking for help.


Specs: OS: Win 10 64bit, CPU: Intel i7 4770 3.4ghz x 4 core(8 threads), GPU: Nvidia GTX 1060 6GB, Ram: 16gig DDR3, Windows on 250gb Samsung Evo 860

Twitter: @SolarPortal
Instagram: @SolarPortal
User avatar
SolarPortal
Skyline Founder
Skyline Founder
 
Posts: 3631
Joined: 29 Jul 2012, 15:56
Location: UK
Skill: 3D Modeller
Skill: 2D Artist
Skill: Programmer
Skill: Level Designer

Re: World Position of GUI Item

Postby Shando » 16 Sep 2017, 13:41

Hi SP,

However, it seems we are lacking the screen pos to world pos. so this will need to be added


I saw the world -> screen commands and thought that screen -> world was missing, or hadn't been documented.

Thanks for the heads up.

Shando
Ryzen 7 4800H 16GB GTX1650 Win 11 64
Love, Hope, Strength http://www.lovehopestrength.co.uk
User avatar
Shando
Skyline Moderator
Skyline Moderator
 
Posts: 560
Joined: 06 Mar 2013, 22:35
Location: Moffat Beach, Queensland
Skill: Programmer
Skill: Scripter
Skill: Level Designer

Re: World Position of GUI Item

Postby SolarPortal » 16 Sep 2017, 16:44

Q) Does it require depth into the scene, or are you need the world space position just in front of camera. As this could be done in c++ by creating a plane in front of the camera and raycasting to that to achieve world space position as if the lens of a camera so to speak or does it need the position of where its placed in the scene.

Also with the current raycasting and it returning zero. This is an absolute zero that is highly unlikely to be that matched on a floating point range while raycasting.. e.g. 0.0001 still will not match to absolute zero 0.0000, this way you can do straight raycast to model level or choose a base depth if it equals absolute zero. So if you can do the click and drag of the gui icon and then use normal raycasting, i cant see why you couldnt place asset directly where needed. :)

With the screen to worldpos, there are 2 types. Screen to world space camera coords .e.g. make a character look at the cursor on screen. Or you can make the character look at a world space ray point based on cursor screen pos... e.g. one looks at camera cursor, the other looks away from camera and at a raycasted surface.
Skyline Game Engine - Lead Dev.
Please provide as much info as possible when asking for help.


Specs: OS: Win 10 64bit, CPU: Intel i7 4770 3.4ghz x 4 core(8 threads), GPU: Nvidia GTX 1060 6GB, Ram: 16gig DDR3, Windows on 250gb Samsung Evo 860

Twitter: @SolarPortal
Instagram: @SolarPortal
User avatar
SolarPortal
Skyline Founder
Skyline Founder
 
Posts: 3631
Joined: 29 Jul 2012, 15:56
Location: UK
Skill: 3D Modeller
Skill: 2D Artist
Skill: Programmer
Skill: Level Designer

Re: World Position of GUI Item

Postby Shando » 16 Sep 2017, 21:56

Hi SP,

Q) Does it require depth into the scene, or are you need the world space position just in front of camera. As this could be done in c++ by creating a plane in front of the camera and raycasting to that to achieve world space position as if the lens of a camera so to speak or does it need the position of where its placed in the scene.


For my purposes, I don't actually require depth as I will spawn the model at a specified depth anyway. I just need some way to get the co-ordinates when their isn't another model for the raycast to hit.

I suppose I could add a giant plane in the background and restrict the available movement/zooming of the board, so that the plane is always guaranteed to be larger than the viewport? That way I could use raycasts??

Thanks

Shando
Ryzen 7 4800H 16GB GTX1650 Win 11 64
Love, Hope, Strength http://www.lovehopestrength.co.uk
User avatar
Shando
Skyline Moderator
Skyline Moderator
 
Posts: 560
Joined: 06 Mar 2013, 22:35
Location: Moffat Beach, Queensland
Skill: Programmer
Skill: Scripter
Skill: Level Designer

Re: World Position of GUI Item

Postby Shando » 09 Oct 2017, 06:43

Hi SP,

Sorry, one final question on this....

How can I get the coordinates (in 3D) of the visible part of an object in the viewport?

image.png
image.png (10.65 KiB) Viewed 15346 times


for example, a command like this:
Code: Select all
minX, minY, maxX, maxY = sky.getObjCoordsInViewport ( objID )


This would make my life so much easier when converting from screen to world as I could then just convert using the screen resolution and these values :)

Thanks

Shando
Ryzen 7 4800H 16GB GTX1650 Win 11 64
Love, Hope, Strength http://www.lovehopestrength.co.uk
User avatar
Shando
Skyline Moderator
Skyline Moderator
 
Posts: 560
Joined: 06 Mar 2013, 22:35
Location: Moffat Beach, Queensland
Skill: Programmer
Skill: Scripter
Skill: Level Designer

Re: World Position of GUI Item

Postby SolarPortal » 09 Oct 2017, 10:44

Is there any reference code you have found while researching to give us a head start..
i'm sure something will be possible based on the bounds of an object and the bounds of the frustum.
Not quite sure of the top of my head though without some form of reference.

Your project is certainly a hard one to visualise without any reference haha :P
Did you try attaching the plane to the camera? Coz that way, it will always be visible regardless of rotation and will be able to raycast to it.
Skyline Game Engine - Lead Dev.
Please provide as much info as possible when asking for help.


Specs: OS: Win 10 64bit, CPU: Intel i7 4770 3.4ghz x 4 core(8 threads), GPU: Nvidia GTX 1060 6GB, Ram: 16gig DDR3, Windows on 250gb Samsung Evo 860

Twitter: @SolarPortal
Instagram: @SolarPortal
User avatar
SolarPortal
Skyline Founder
Skyline Founder
 
Posts: 3631
Joined: 29 Jul 2012, 15:56
Location: UK
Skill: 3D Modeller
Skill: 2D Artist
Skill: Programmer
Skill: Level Designer

Re: World Position of GUI Item

Postby Shando » 09 Oct 2017, 12:55

Hi SP,

Please ignore the ramblings of an old man :roll:

I forgot to normalise the vector!!!!!

All is looking pretty good now, as I can drag n drop a model :mrgreen:

Thanks and sorry for wasting your precious time :oops:

Shando

EDIT: I have come across something a little strange with the Raycast...

I have a Scene that has the following entities:

Default Light - Directional @ -4, 10, 10
Default Empty Node @ 0, 0, 0
Camera - 2D Fixed @ 0, 10, 0
Plane @ 0, 0, -200

When I spawn in the new model, everything is fine until I get approx. halfway down the screen when it suddenly starts jumping about? According to my messages it looks like the ray is hitting something at approx. -22 in the Z axis:
Code: Select all
Ray: -7.430278301239 | 20.194133758545 | -199.80001831055
--> Ray: -0.77889364957809 | 10.900088310242 | -22.907981872559
Ray: -5.7319264411926 | 15.106592178345 | -199.80000305176
Ray: -5.5196380615234 | 14.668340682983 | -199.79998779297
Ray: -5.5196385383606 | 13.829944610596 | -199.80000305176
Ray: -5.0950436592102 | 12.972494125366 | -199.80000305176
Ray: -5.0950436592102 | 12.972494125366 | -199.80000305176
--> Ray: -0.52799093723297 | 10.071084976196 | -22.587203979492
Ray: -4.0335750579834 | 7.6563024520874 | -199.79998779297


My model is spawned in at -20 in the Z axis, so I'm just wondering if it's hitting a part of the model for some reason?

EDIT 2: After some more tests, it is *definitely* hitting the Model :(

Is there a way to set the starting point of the Raycast or to ignore the 1st hit object?
Ryzen 7 4800H 16GB GTX1650 Win 11 64
Love, Hope, Strength http://www.lovehopestrength.co.uk
User avatar
Shando
Skyline Moderator
Skyline Moderator
 
Posts: 560
Joined: 06 Mar 2013, 22:35
Location: Moffat Beach, Queensland
Skill: Programmer
Skill: Scripter
Skill: Level Designer

Re: World Position of GUI Item

Postby SolarPortal » 09 Oct 2017, 14:07

which ray command are you using?

because we do use query flags to stop raycasting to itself, but depending on which raycast your using then it may or may not be implemented... :)

Good that your up and running though haha :P and no worries about the time wasted.. just had me a bit baffled lol :P
Skyline Game Engine - Lead Dev.
Please provide as much info as possible when asking for help.


Specs: OS: Win 10 64bit, CPU: Intel i7 4770 3.4ghz x 4 core(8 threads), GPU: Nvidia GTX 1060 6GB, Ram: 16gig DDR3, Windows on 250gb Samsung Evo 860

Twitter: @SolarPortal
Instagram: @SolarPortal
User avatar
SolarPortal
Skyline Founder
Skyline Founder
 
Posts: 3631
Joined: 29 Jul 2012, 15:56
Location: UK
Skill: 3D Modeller
Skill: 2D Artist
Skill: Programmer
Skill: Level Designer

Re: World Position of GUI Item

Postby Shando » 09 Oct 2017, 21:31

Hi SP,

I'm using the raycastToCursor command:
Code: Select all
x, y, z = sky.raycastToCursor ( )


It's not a major problem at the moment, as I can probably code around it for now and the jumping is not too bad as long as you keep moving the mouse ;)

Thanks again.

Shando
Ryzen 7 4800H 16GB GTX1650 Win 11 64
Love, Hope, Strength http://www.lovehopestrength.co.uk
User avatar
Shando
Skyline Moderator
Skyline Moderator
 
Posts: 560
Joined: 06 Mar 2013, 22:35
Location: Moffat Beach, Queensland
Skill: Programmer
Skill: Scripter
Skill: Level Designer

Re: World Position of GUI Item

Postby SolarPortal » 09 Oct 2017, 21:56

yeah, that function doesnt pass the query data.

But you could try this:
When your placing the object, then use this function:

Code: Select all
    entity.setQueryFlag_DoNotSelect(obj)


then do your raycasts. After that if you need an entity to be raycast-able... making words here lol :P
then you can use:

Code: Select all
    entity.setQueryFlag_Model(obj)


which will re-engage raycasts on that models surface. Model is what everything has by default.

If you need a custom query flag for an entity, then you can use:
Code: Select all
entity.setQueryFlag(obj, flag) -- flag is an int

Note sure how usable it would be though if the raycast cannot change query.

Hope this helps :D
Skyline Game Engine - Lead Dev.
Please provide as much info as possible when asking for help.


Specs: OS: Win 10 64bit, CPU: Intel i7 4770 3.4ghz x 4 core(8 threads), GPU: Nvidia GTX 1060 6GB, Ram: 16gig DDR3, Windows on 250gb Samsung Evo 860

Twitter: @SolarPortal
Instagram: @SolarPortal
User avatar
SolarPortal
Skyline Founder
Skyline Founder
 
Posts: 3631
Joined: 29 Jul 2012, 15:56
Location: UK
Skill: 3D Modeller
Skill: 2D Artist
Skill: Programmer
Skill: Level Designer


Return to UI

Who is online

Users browsing this forum: No registered users and 2 guests