Sikuli Overview

Sikuli is a Desktop Automation library. This library “sikulixapi.jar” can be imported to any Java Selenium project to automate any desktop app or popup.
Sikuli uses “Screen” Class and it has lots of methods to support GUI interactions.E.g. Screen s = new Screen();s.click(“imageOfButton.png”);Note: Store imageOfButton.png at java root project folder.
Similarly you could do type, doubleClick, find etc methods provided by Screen class.
Sikuli automation is very simple and can be used to replace many day today tasks on outlook, windows software center installation. With that said sikuli can replace some of the robotic process automation done by many tools like Automation Anywhere, UiPath etc.

What is Sikuli Screen, Region, Pattern ?

Sikuli Screen is a class that has lots methods like click, double click, find etc to interact desktop GUI images.

Region class is to target a rectangular area location on GUI using x,y coordinates and width and height to locate the rectangular area.

Pattern class used for image level scanning support

Sikuli Architecture:

Sikuli scripts have Java library that takes care of mouse and keyboard actions using java.awt.Robot. Apart from this, Sikuli using OpenCV library which is C++ engine connected to Java via JNI. OpenCV is a computer Vision and machine learning library used for image recognition.

What is JNI

JNI is Java Native interface, it is java programming interface used to bridge the Java and other non java programming language like C, C++ application/programming frameworks