Game UI

core.game_ui finds the game's own windows and the controls inside them: buttons, check boxes, text boxes, drop-down lists, tabs, sliders, scroll bars and text labels. You can read what each control shows and operate it the way a player does, by clicking, pressing keys, typing, ticking boxes, choosing items and moving sliders.

Windows are returned as Window objects and controls as Control objects. Some windows draw their items themselves instead of using controls, such as the zone tiles of the Monster Park window. Those items are returned as a List object, and window:click_at() clicks a window that draws its own contents.

To react to what happens in game windows, use the on_ui_event, on_window_open and on_window_close callbacks.

#Finding windows

#core.game_ui.get_windows() -> table

Available on GMS and CWGMSCW

Returns every open top-level window as an array of Window objects. Windows nested inside another window are returned by window:get_child_windows() instead.

#core.game_ui.find_window(filter: table | function | nil) -> Window | nil

Available on GMS and CWGMSCW

Returns the first open window that matches filter, or nil if none does. filter is a table of the fields below, all of which must match, or a function that receives each Window and returns true to accept it. Without a filter, any window matches.

Text matching ignores case. Raises an error when filter is not a table or a function, a field has the wrong type, kind is not one of the kinds listed under get_kind(), ui_type is not a whole number, or a text field is longer than 256 characters.

local shop = core.game_ui.find_window({ kind = "shop" })

local wide = core.game_ui.find_window(function(window)
    local size = window:get_size()
    return size ~= nil and size.width > 500
end)

#core.game_ui.find_windows(filter: table | function | nil) -> table

Available on GMS and CWGMSCW

Same as find_window, but returns every matching window as an array, which is empty when none match.

#core.game_ui.get_window(handle: number) -> Window | nil

Available on GMS and CWGMSCW

Returns the window with this handle while it is open, or nil.

#core.game_ui.get_control(handle: number) -> Control | nil

Available on GMS and CWGMSCW

Returns the control with this handle while it exists, or nil.

#core.game_ui.get_focused_control() -> Control | nil

Available on GMS and CWGMSCW

Returns the control that has keyboard focus, such as the text box you are typing in, or nil when no control has it.

#Window

#window:is_valid() -> boolean

Available on GMS and CWGMSCW

Returns true while the window is open.

#window:get_handle() -> number | nil

Available on GMS and CWGMSCW

Returns the number that identifies this window while it is open.

#window:get_kind() -> string | nil

Available on GMS and CWGMSCW

Returns what sort of window this is:

#window:get_ui_type() -> number | nil

Available on GMS and CWGMSCW

Returns the number the game gives this kind of window, or nil for windows that don't have one. Every ui_window has one, and some windows of the other kinds do too. A window has the same number every time it opens, which makes the number a reliable way to recognise it. Numbers can change when the game updates.

To find a window's number, open the window and list what is open:

for _, window in ipairs(core.game_ui.get_windows()) do
    core.log(string.format("%s  ui_type=%s  %s", window:get_kind(),
        tostring(window:get_ui_type()), window:get_resource_path() or "-"))
end

#window:get_resource_path() -> string | nil

Available on GMS and CWGMSCW

Returns the path of the window's artwork in the game's UI data, or nil when it has none. Like get_ui_type(), it tells windows apart. The resource filter matches part of it.

#window:get_position() -> table | nil

Available on GMS and CWGMSCW

Returns the top-left corner of the window on the game screen as a table with x and y fields.

#window:get_size() -> table | nil

Available on GMS and CWGMSCW

Returns the window's size as a table with width and height fields.

#window:is_enabled() -> boolean | nil

Available on GMS and CWGMSCW

Returns whether the window accepts input.

#window:get_controls() -> table | nil

Available on GMS and CWGMSCW

Returns the window's own controls as an array of Control objects. Controls inside child windows are not included; window:find_controls() searches both.

#window:get_child_windows() -> table | nil

Available on GMS and CWGMSCW

Returns the windows nested inside this one as an array of Window objects.

#window:get_list() -> List | nil

Available on GMS and CWGMSCW

Returns the grid of items the window draws itself, such as the zone tiles of the Monster Park window, as a List. Returns nil when the window has none. Items that are controls, such as rows of buttons, are found with find_controls() instead.

#window:find_control(filter: table | function | nil) -> Control | nil

Available on GMS and CWGMSCW

Returns the first control that matches filter, or nil if none does. It searches the window's own controls first, then those of its child windows, up to four levels deep. filter is a table of the fields below, all of which must match, or a function that receives each Control and returns true to accept it.

Raises an error when filter is not a table or a function, a field has the wrong type, type is not one of the types listed under get_type(), id is not a whole number, or text is longer than 256 characters.

local window = core.game_ui.find_window({ kind = "npc_chat" })
if window then
    for _, control in ipairs(window:find_controls({ visible = true })) do
        core.log(string.format("%s  id=%d  %s", control:get_type(), control:get_id(), control:get_text() or ""))
    end
end

#window:find_controls(filter: table | function | nil) -> table | nil

Available on GMS and CWGMSCW

Same as find_control, but returns every matching control as an array.

#window:click_button(id: number) -> boolean, string?

Available on GMS and CWGMSCW

Clicks the button with this id in the window or one of its child windows, the same way as control:click(), including when the click is sent. Returns false, error_message when the window has closed, has no button with that id, or the button is hidden or disabled.

#Control

#control:is_valid() -> boolean

Available on GMS and CWGMSCW

Returns true while the control exists.

#control:get_handle() -> number | nil

Available on GMS and CWGMSCW

Returns the number that identifies this control while it exists.

#control:get_id() -> number | nil

Available on GMS and CWGMSCW

Returns the control's id within its window. A window's layout fixes the ids of its controls, so a button has the same id every time its window opens.

#control:get_type() -> string | nil

Available on GMS and CWGMSCW

Returns what sort of control this is:

#control:get_window() -> Window | nil

Available on GMS and CWGMSCW

Returns the window that contains the control.

#control:get_text() -> string | nil

Available on GMS and CWGMSCW

Returns the text the control shows: the label of a button or check box, the contents of a text box, the text of a label, the selected item of a drop-down list or the name of the selected tab. Returns nil for other controls.

#control:is_enabled() -> boolean | nil

Available on GMS and CWGMSCW

Returns whether the control accepts input.

#control:is_visible() -> boolean | nil

Available on GMS and CWGMSCW

Returns whether the control is shown.

#control:get_position() -> table | nil

Available on GMS and CWGMSCW

Returns the top-left corner of the control on the game screen as a table with x and y fields.

#control:get_size() -> table | nil

Available on GMS and CWGMSCW

Returns the control's size as a table with width and height fields.

#control:is_checked() -> boolean | nil

Available on GMS and CWGMSCW

Returns whether a check box is ticked or a button is toggled on. Returns nil for other controls.

#control:is_read_only() -> boolean | nil

Available on GMS and CWGMSCW

Returns whether a text box is read-only, or nil for other controls. set_text() refuses read-only boxes.

#control:is_password() -> boolean | nil

Available on GMS and CWGMSCW

Returns whether a single-line text box hides what is typed in it, or nil for other controls. get_text() still returns its contents.

#control:get_value() -> number | nil

Available on GMS and CWGMSCW

Returns the position of a slider or scroll bar, or nil for other controls.

#control:get_range() -> number, number | nil

Available on GMS and CWGMSCW

Returns the lowest and highest positions a slider or scroll bar accepts, or nil for other controls.

#control:get_items() -> table | nil

Available on GMS and CWGMSCW

Returns the items of a drop-down list or the names of a row of tabs, as an array of strings in display order. Returns nil for other controls.

#control:get_selected() -> number | nil

Available on GMS and CWGMSCW

Returns the 0-based index of the selected item or tab, or nil for other controls.

#List

A List is a grid of items that a window draws itself. Items are numbered from 0, left to right and then top to bottom, across the whole list, including items on other pages or scrolled out of view. When the window reloads its items, the old List object stops being valid; call window:get_list() again.

#list:is_valid() -> boolean

Available on GMS and CWGMSCW

Returns true while the window still shows this list.

#list:get_window() -> Window | nil

Available on GMS and CWGMSCW

Returns the window that shows the list.

#list:get_count() -> number | nil

Available on GMS and CWGMSCW

Returns how many items the list has, including those not on screen.

#list:get_selected() -> number | nil

Available on GMS and CWGMSCW

Returns the 0-based index of the selected item, or nil when no item is selected.

#Actions

Actions work like a player's input, and the window reacts as it would to a real click, key press or keystroke. Each returns true when the input was accepted, or false, error_message when nothing was done, for example because the control is hidden, disabled, gone, or ignores that kind of input.

Clicks and key presses (click, right_click, press_key, click_at, right_click_at and click_button) are sent once the callback that made them returns, in the order you made them. The window reacts on a later tick, so read its new state or wait for on_ui_event and on_window_open there. Typing, ticking check boxes, moving sliders and selecting items take effect before the call returns.

While a dialog box is open and waiting for an answer, such as a "complete all?" confirmation, your callbacks keep running. Answer it by clicking its own buttons: it is a window of kind "dialog" or "message_box". Input to any other window is refused with false, error_message until the dialog box closes, as it is for your own clicks.

#control:click() -> boolean, string?

Available on GMS and CWGMSCW

Clicks the middle of the control with the left mouse button.

#control:right_click() -> boolean, string?

Available on GMS and CWGMSCW

Clicks the middle of the control with the right mouse button.

#control:press_key(key: number) -> boolean, string?

Available on GMS and CWGMSCW

Presses and releases a key on the control. key is a Windows virtual-key code from 1 to 254. For example, 0x0D (Enter) in a text box does what pressing Enter while typing in it does.

#control:set_text(text: string) -> boolean, string?

Available on GMS and CWGMSCW

Replaces the contents of a text box by clearing it and typing text one character at a time. The box applies the same rules as it does to your typing, so it may shorten the text, skip characters it doesn't take (such as letters in a number box) or lower a number to its limit. Read get_text() afterwards to see what was entered. text can be up to 4096 characters. Characters outside the Western European (Windows-1252) set become the closest match or ?, and line breaks are only typed into multi-line boxes. Returns false, error_message for controls that are not text boxes and for read-only boxes.

#control:set_checked(checked: boolean) -> boolean, string?

Available on GMS and CWGMSCW

Ticks or clears a check box by clicking it when it isn't already in that state. Returns false, error_message when the control is not a check box or the click did not change it. Some check boxes work like option buttons and can't be cleared by clicking them.

#control:set_value(value: number) -> boolean, string?

Available on GMS and CWGMSCW

Moves a slider or scroll bar to value, kept within get_range(). The window reacts as it does when you drag the slider or scroll bar there and let go. Returns false, error_message for other controls.

#control:select(item: number | string) -> boolean, string?

Available on GMS and CWGMSCW

Selects an item in a drop-down list or a tab in a row of tabs, by 0-based index or by its exact text (ignoring case). The window reacts as it does when you choose it yourself. Returns false, error_message for other controls, an index out of range, text that matches no item, or an item the game won't let you choose.

#window:click_at(x: number, y: number) -> boolean, string?

Available on GMS and CWGMSCW

Clicks the window with the left mouse button at x, y, measured in pixels from its top-left corner, the same space as get_size(). The click goes to the window itself, which is what a window that draws its own contents reacts to. It does not reach controls drawn on the window; click those with control:click(). Returns false, error_message when the point is outside the window, or the window is disabled or ignores clicks.

#window:right_click_at(x: number, y: number) -> boolean, string?

Available on GMS and CWGMSCW

Same as click_at, with the right mouse button.

#window:press_key(key: number) -> boolean, string?

Available on GMS and CWGMSCW

Presses and releases a key on the window itself, as when the window has keyboard focus. key is a Windows virtual-key code from 1 to 254. For example, 0x0D (Enter) does what pressing Enter in the window does. The key does not reach the window's controls; use control:press_key() for those. Returns false, error_message when the window is disabled or ignores key presses.

#list:select(index: number) -> boolean, string?

Available on GMS and CWGMSCW

Selects item index (0-based) the way a player does with the arrow keys. The selection moves one item at a time with the window's own Left and Right keys until it reaches index, and the window turns pages or scrolls as it does for your key presses. Returns false, error_message for an index out of range, a disabled window, or when the window stops moving the selection, for example at an item that can't be selected. The selection may then have moved part of the way.

#Example

-- Type an amount into the focused text box and confirm it with Enter.
local box = core.game_ui.get_focused_control()
if box and (box:get_type() == "edit" or box:get_type() == "multiline_edit") then
    local ok, err = box:set_text("100")
    if ok then
        box:press_key(0x0D)
    else
        core.log_error(err)
    end
end

-- Report every window that opens and every button clicked in it.
core.register_callback("on_window_open", function(window)
    core.log("opened " .. window:get_kind() .. " " .. (window:get_resource_path() or ""))
end)

core.register_callback("on_ui_event", function(event)
    if event.type == "click" and event.window then
        core.log("clicked button " .. event.control_id .. " in " .. event.window:get_kind())
    end
end)

-- Enter the third zone listed in the open Monster Park window.
local park = core.game_ui.find_window({ resource = "UIMonsterPark" })
local zones = park and park:get_list()
if zones then
    local ok, err = zones:select(2)
    if ok then
        park:press_key(0x0D)
    else
        core.log_error(err)
    end
end