Python - Automate Mouse and Keyboard Clicks!

/

Are you too lazy to click?

I know I am. Automatic key presses and mouse clicks can come in handy for a plethora of reasons.
You can automate you clicking game, post tweets automatically, do Excel activities, automate tasks that don’t have an API, scare your coworkers - possibilities are endless.

There is almost always a better way - as automating clicks is quite inefficient. For example I learnt much of this automating Twitter posts, but their API can do the same thing much much quicker. For example you can get banned in 5 minutes of API use, instead of 5 weeks of auto clicking use!

So the MVP point of this, is that it is easy and quick. Yes, there will be a better way, but it could take twice as long and not be as fun. I’ve used this before to automate Excel tasks - while a macro would be better this was 10x quicker to code and was perfect for the one off task I needed to automate.

time v productivity gain.png

Here is the code

This Python library has great instructions - most not here you can easily find by Googling.
https://pyautogui.readthedocs.io/en/latest/

You work it by telling the mouse where to go on your screen. So first of all, you need to find what coordinates to add in. 

import pyautogui as pg
import time

time.sleep(5) #Time for you to move your mouse
print(pg.size()) #Monitor Size
print(pg.position()) #Mouse pointer location

It will output something like: Size(width=3840, height=2160) Point(x=8048, y=588). I have two monitors, which is why the X is quite large.

To move your mouse to the point, click on it, then write something:

import pyautogui as pg
import time

# time.sleep(5) #Time for you to move your mouse
# print(pg.size()) #Monitor Size
# print(pg.position()) #Mouse pointer location



pg.moveTo(8048, 588, 1) #X,Y,Seconds
time.sleep(2)
pg.click(8048, 588) #X,Y
time.sleep(2)
pg.typewrite('Excellen')

The time.sleep is just to let things load - especially important if you are opening browsers/webpages etc.
You also don’t need the moveTo command, click would just work. However I like the realism!

Here’s an example I used to help me add the Excel formulas every 18 rows. You also see how to command keys like ‘backspace’ etc. 

import pyautogui as pg

#Make sure you are clicked in starting cell!
StartRow = 7
Formula = r"=TRANSPOSE('Assign Jobs'!MI"+str(StartRow)+r":NA"+str(StartRow)+r")"
# print(Formula)
#34
for x in range (59):

    #Add Formula in formula bar
    pg.click(572, 341)
    pg.typewrite(Formula)
    pg.press('enter')

    #Move down 18 rows
    pg.press('down', presses=18)
    StartRow = StartRow + 1
    Formula = r"=TRANSPOSE('Assign Jobs'!MI"+str(StartRow)+r":NA"+str(StartRow)+r")"

Go to the website above to learn about combination like pg.hotkey('ctrl', 'v') - most things are possible!
It even has a ML module, where you can search for images within a confidence interval. I used it to find the Twitter follow button which changes location. You just need to screenshot what it looks like!

# Finds follow button
                FindFollowButton = pg.locateOnScreen('follow.png', confidence=.9)

                if FindFollowButton == None:
                    print('cant find it - skip')
                    pg.moveTo(96, 531, 0.5)
                    pg.click(96, 531)


                else:
                    print('found it - follow')
                    ButtonLocation = pg.center(FindFollowButton)
                    time.sleep(0.5)
                    pg.click(ButtonLocation)
                    time.sleep(0.5)
                    pg.moveTo(96, 531, 0.5)

Finally, you can use it to screenshot!

# For screenshots, left,top,width,height
# im = pg.screenshot('mmmm.png',region =(600,0,600,500))

There are countless more cool features that you can find on their website. As I mentioned, not super efficient, but it’s quick and easy. Excellent for mundane tasks! Couple it with my Macro Scheduling tutorial and you can even further automate!
Let me know if you have any cool uses for this!