{"id":2388,"date":"2024-09-03T04:28:03","date_gmt":"2024-09-03T04:28:03","guid":{"rendered":"https:\/\/writingagame.com\/?p=2388"},"modified":"2025-01-25T19:18:36","modified_gmt":"2025-01-25T19:18:36","slug":"chapter-11-interactivity","status":"publish","type":"post","link":"https:\/\/writingagame.com\/index.php\/2024\/09\/03\/chapter-11-interactivity\/","title":{"rendered":"Chapter 11. Interactivity"},"content":{"rendered":"\n<p><\/p>\n\n\n\n<p>Now we&#8217;ll try to implement game objects&#8217; reaction on our own manual input (mouse for instance). The main part is how to determine which game object the mouse is pointing at. We&#8217;ll use the same approach as in previous chapter. Unlike in collision detection we&#8217;ll calculate object&#8217;s sizes\/position not in world X-Z plane, but in X-Y screen plane.<\/p>\n\n\n\n<p>Source code: <a href=\"https:\/\/github.com\/bkantemir\/_wg_411\">https:\/\/github.com\/bkantemir\/_wg_411<\/a><\/p>\n\n\n\n* Just in case, executable demo (Windows) <a href=\"https:\/\/writingagame.com\/img\/b04\/c11\/DemoTrain.zip\" download=\"\">here<\/a>.\n\n\n\n<p>Windows Defender will complain, but that&#8217;s to be expected, since this is not an approved source of executable files.<\/p>\n\n\n\n<p><\/p>\n\n\n\n<p>Now, back to the topic: Start Visual Studio and open <em>CPP\/a996rr\/pw\/pw.sln<\/em> project.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\">Case 1: 3D objects on the screen<\/h2>\n\n\n\n<p>Let&#8217;s say we want to grab and drag locomotive here:<\/p>\n\n\n\n<p><\/p>\n\n\n\n<p><img decoding=\"async\" src=\"https:\/\/writingagame.com\/img\/b04\/c11\/01.jpg\"><\/p>\n\n\n\n<p><\/p>\n\n\n\n<p>How program supposed to detect is mouse on the locomotive or not?<\/p>\n\n\n\n<p>Well, the first thing that comes to mind is to check bounding box:<\/p>\n\n\n\n<p><\/p>\n\n\n\n<p><img decoding=\"async\" src=\"https:\/\/writingagame.com\/img\/b04\/c11\/02.jpg\"><\/p>\n\n\n\n<p><\/p>\n\n\n\n<p>Obviously, being inside the bounding box does not automatically mean hitting the object (as shown above).<\/p>\n\n\n\n<p>Hitting the projection of original (source) bounding box &#8211; YES, that would work:<\/p>\n\n\n\n<p><\/p>\n\n\n\n<p><img decoding=\"async\" src=\"https:\/\/writingagame.com\/img\/b04\/c11\/03.jpg\"><\/p>\n\n\n\n<p><\/p>\n\n\n\n<p>However, checking <em>such <\/em>arbitrarily oriented rectangle is a rather challenging and somewhat cumbersome task. Instead, I am calculating the longest axis projection (I called it &#8220;CHORDA&#8221;) and a radius R as the shorter axis bisected:<\/p>\n\n\n\n<p><\/p>\n\n\n\n<p><img decoding=\"async\" src=\"https:\/\/writingagame.com\/img\/b04\/c11\/04.jpg\"><\/p>\n\n\n\n<p><\/p>\n\n\n\n<p>So, now the task comes down to calculating the distance from the cursor to the chorda. If less than chorda&#8217;s R &#8211; then we are in.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Implementation:<\/h2>\n\n\n\n<ol class=\"wp-block-list\">\n<li>In the very beginning, in the <em>main.cpp<\/em> we&#8217;re calling <em>CPP\/p_windows\/input.cpp -&gt; initMouseInput()<\/em> to initiate mouse input callbacks. Plus to upload several different cursors. This function is <em>Windows-specific<\/em>, Android will require different implementation. These callbacks will record mouse events into our own <em>TouchScreen::touchScreenEvents<\/em> array for consequent processing.<\/li>\n\n\n\n<li>Original (source) bounding boxes: when loading models in <em>ModelLoader::loadModelStandard()<\/em> we are calling <em>buildGabaritesFromDrawJobs() <\/em>function that fills structure <em>gabaritesOnLoad<\/em> in the <em>SceneSubj <\/em>object.<\/li>\n\n\n\n<li>During the game, in each frame we need to calculate models bounding boxes&#8217; screen projections: in the<em> TheApp::drawFrame()<\/em>, when scanning our <em>SceneSubjs<\/em>, I&#8217;m calling <em>Gabarites::fillGabarites()<\/em>  (line 290) to fill out <em>SceneSubj&#8217;s  gabaritesOnScreen<\/em> structure for consequent use.<\/li>\n\n\n\n<li>Plus we&#8217;re calling <em>ui\/TouchScreen::getInput()<\/em> (line 217), which checks cursor position, finds matching <em>SceneSubj <\/em>(if any), and decides which action to call.<\/li>\n<\/ol>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<p><\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Case 2: buttons (2D)<\/h2>\n\n\n\n<p>Even easier. Since they are 2D, don&#8217;t even need to pre-calculate &#8220;chords&#8221; or bounding boxes. If distance from cursor is less than button radius &#8211; then hit. Structure is much simpler than <em>SceneSubj<\/em>, so they are organized in a separate class <em>UISubj<\/em>. Still there is a set of interactivity-related functions common with <em>SceneSubjs<\/em>, such as:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>isClickable()<\/li>\n\n\n\n<li>iisDraggable()<\/li>\n\n\n\n<li>isResponsive()<\/li>\n\n\n\n<li>onFocus()<\/li>\n\n\n\n<li>onFocusOut()<\/li>\n\n\n\n<li>onLeftButtonDown()<\/li>\n\n\n\n<li>onLeftButtonUp()<\/li>\n\n\n\n<li>onDrag()<\/li>\n\n\n\n<li>onClick()<\/li>\n\n\n\n<li>onDoubleClick()<\/li>\n\n\n\n<li>onLongClick()<\/li>\n<\/ul>\n\n\n\n<p>To make both 2D <em>UISubjs <\/em>and 3D <em>SceneSubjs <\/em>equally accessible in <em>TouchScreen <\/em>class, this functionality is declared separately in a new <em>ScreenSubj <\/em>class,  a <strong>parent <\/strong>for both <em>UISubj <\/em>and <em>SceneSubj <\/em>classes.<\/p>\n\n\n\n<p><\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\">Case 3: dragging background (a game table)<\/h2>\n\n\n\n<p>The challenge here is to figure out 3D point on the table corresponding to 2D cursor&#8217;s screen position. Solution is to build 3D line (ray) from camera to the table passing through screen cursor position. Implemented in <em>TheTable::getCursorAncorPointTable()<\/em>.<\/p>\n\n\n\n<p>Key steps:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Calculate cursor screen position in OpenGL terms (in -1 to 1 range) for x and y, and +1 for z (most distant point)<\/li>\n\n\n\n<li>Calculate inverted matrix for mainCamera&#8217;s View-Projection matrix<\/li>\n\n\n\n<li>Apply this inverted matrix to GL-formatted cursor position. Resulting vector is world coordinates of the most distant point of the ray from camera through the cursor.<\/li>\n\n\n\n<li>Having camera&#8217;s coordinates and ray&#8217;s most distant point &#8211; calculate 3D line equation<\/li>\n\n\n\n<li>Find ray&#8217;s intersection point with table&#8217;s surface. Bingo!<\/li>\n<\/ul>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\">Android<\/h2>\n\n\n\n<p>Start Android Studio and open <em>CPP\/a996rr\/pa<\/em> project.<\/p>\n\n\n\n<p>The difference from Windows solution is how we read the input. Instead of mouse events, we will obviously use touch events. We don&#8217;t even need special listeners here. <em>CPP\/p_android\/platform.cpp -&gt; myPollEvents()<\/em> receives all the related events directly. Then we need to convert them to our TouchScreen::touchScreenEvents array, the rest is handled by the shared code described above.<\/p>\n\n\n\n<p><\/p>\n\n\n\n<p>One issue that deserves special attention:<\/p>\n\n\n\n<p>Of course, this solution heavily relies on touch events like <em>AMOTION_EVENT_ACTION_DOWN<\/em> and <em>AMOTION_EVENT_ACTION_UP<\/em> (finger on and out of screen). Don&#8217;t know who to blame: Android, GameActivity, Samsung (in my case) or whoever makes screens for them, but anyway: these key events are often missed. So, I added a special extra code to track this: <em>CPP\/p_android\/platform.cpp -&gt; myPollEvents()<\/em>, from line 178.<\/p>\n\n\n\n<p>The idea is:<\/p>\n\n\n\n<p>If we have some touch-screen events when assuming that we don&#8217;t have any fingers on screen, it means that we (device) overlooked <em>AMOTION_EVENT_ACTION_DOWN<\/em>.<\/p>\n\n\n\n<p>And vice-versa: we assuming that we DO have a finger on screen, but there are no any incoming events for a while. Means &#8211; perhaps <em>AMOTION_EVENT_ACTION_UP<\/em> overlooked.<\/p>\n\n\n\n<p><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>In newer Android Studio this problem was solved, new function <em>android_input_buffer* inputBuffer = android_app_swap_input_buffers(pAndroidApp);<\/em> <\/li>\n<\/ul>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<p><\/p>\n\n\n\n<p><\/p>\n","protected":false},"excerpt":{"rendered":"<p class=\"mb-2\">Now we&#8217;ll try to implement game objects&#8217; reaction on our own manual input (mouse for instance). The main part is how to determine which game object the mouse is pointing at. We&#8217;ll use the same approach as in previous chapter. Unlike in collision detection we&#8217;ll calculate object&#8217;s sizes\/position not in world X-Z plane, but in [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":2412,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[4],"tags":[],"class_list":["post-2388","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-cross-platform-3d"],"_links":{"self":[{"href":"https:\/\/writingagame.com\/index.php\/wp-json\/wp\/v2\/posts\/2388","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/writingagame.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/writingagame.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/writingagame.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/writingagame.com\/index.php\/wp-json\/wp\/v2\/comments?post=2388"}],"version-history":[{"count":41,"href":"https:\/\/writingagame.com\/index.php\/wp-json\/wp\/v2\/posts\/2388\/revisions"}],"predecessor-version":[{"id":2772,"href":"https:\/\/writingagame.com\/index.php\/wp-json\/wp\/v2\/posts\/2388\/revisions\/2772"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/writingagame.com\/index.php\/wp-json\/wp\/v2\/media\/2412"}],"wp:attachment":[{"href":"https:\/\/writingagame.com\/index.php\/wp-json\/wp\/v2\/media?parent=2388"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/writingagame.com\/index.php\/wp-json\/wp\/v2\/categories?post=2388"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/writingagame.com\/index.php\/wp-json\/wp\/v2\/tags?post=2388"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}