Learn how to click when image appear, wait for image to disappear

Image Detection in a Nutshell
How it works

First, you choose an area on the screen called a Region. A smaller image or text to detect in that area which is called a Template.

The Template is slided across the Region to find similar image (or text is read using OCR and compare in the case that the Template is a text).

Each part in the Region will then be scored on how similar it is to the Template. This score will be called Match Score

If any places that has a Match Score higher than the Min Score, a Match is found. Actions can then be taken on the match(es) depend on what you've choose.

If a match is not found then the whole operation is repeated until the Timeout expired.

If you select one of the loop option, the whole operation will be repeated

How to Create
  • Method 1: Using the icon in Toolbar or Bottombar.
  • Method 2: Using the Region class in code.

A Region is just an area on the screen where you want to look for your Template.

You can Drag the Region to change its location, Drag the small triangle to change its size.

You should be a little generous with the Region size so it would work on all devices size.

Region Scaling
Region Scaling

Region is automatically scaled between different screen size. There are 3 scale mode:

  • Fixed: The position and size is scaled with device size. Mostly used for elements in middle
  • Dynamic: The position and size are dynamicly calculated by percentage of the current screen. Mostly used for elements that is aligned to the edge
  • Sticky: Similar to Fixed scaling but you can choose the edges that region align (1.4.0+)
Region Preview
Region Preview

You can use the icon to preview region in different screen ratio

Make use of this to make sure your macro work across different screen size

Image Detection Popup

Template is the thing you want to search for in the Region.

This is only about image Template. Text Recognition and Color Detection has its own section.

Use to take a screenshot.

Select an already taken Template from Resources

Use Advanced if you want to enter the image name as a string directly or a variable holding the image name.

1.4.0+: You can select a Color Resource as the Template. This will perform a color search in the Region.

Image Detection Actions

4 actions are currently available when a Template is found or not found.

  • Click best match: Click on the most likely Match. This will probably be the most used action.
  • Click multiple matches: Click all the Matches in the Region.
  • Wait till appear: Simply wait for the Template to appear. Mostly use with Callbacks.
  • Wait till vanish: Wait for Template to disappear. The operation ends immediately if the Template is not there to begin with. Mostly use with Callbacks.

You can access more options in code. Check Region class.

Find Parameters
Find Parameters

You can change some parameters that alter the find operation.

Timeout period specify how long the search should last. -1 for forever

Set the minimum Match Score called Min Score. Anything above this score is considered a Match

Set the Scan rate. 3 for 3 scans per second. 0.5 for half scan a second or 1 scan per 2 seconds. If the wait period is long then you should lower the Scan rate to conserve battery.

1.4.0+: When Color is the Template then Min Score should have high score to prevent false positive. Generally > 0 .95 is good

Click Parameters
Click Parameters

If the Action is Click best match or Click multiple matches then you can control how that click is done.

All parameters is the same as Click.

Loop Parameters

New in ver 1.1.0. See the flow chart above.

When use loop. Image Detection will run again from the start

  • None: No Loop, Image Detection runs once.
  • Loop when success: Loop back when Image Detection is successful, end when fail.
  • Loop when fail: Loop back when Image Detection fail, end when success.
  • Loop always: Always loop back regardless of Image Detection's result.
On Success, On Fail
On Success, On Fail

New in ver 1.3.0. See the flow chart above.

On Success will run when the image is found (click best match, click multple matches, wait till appear) or image is disappeared (wait till vanish)

On Fail will run when the opposite happens or timeout occured

On Success and On Fail are a Group so you can add any actions that a group can support including another Image Detection for unlimited branching

if-else logic can be implemented easily

Code Equivalent

    //Create a Region
    var region = Region(100, 100, 500, 500)

    //Click on the template
    var match ="template name", FParam.timeout(3000).sRate(2).mScore(0.8))

    if (match) {
    } else {

© 2024 - Macrorify by KoK-CODE