Thread: DirectRobot - A fast Java Robot that allows for screen recording

Page 1 of 3 123 LastLast
Results 1 to 10 of 27
  1. #1 DirectRobot - A fast Java Robot that allows for screen recording 
    Registered Member
    Killer 99's Avatar
    Join Date
    Dec 2007
    Posts
    1,484
    Thanks given
    171
    Thanks received
    508
    Rep Power
    414
    I actually wrote this for my RDP (Remote Desktop Program) that I was writing and I thought I'd share it with ya'll so you can do something fun with it.

    Why did I make this? Because java.awt.Robot is very indirect, has a lot of added garbage esp. when dealing with screen captures. Specifically to use the screen capture method for an RDP would be a nightmare, everytime you call the method it first has to create a pixel array to call the method to grab the screen pixels, then it has to make it into a BufferedImage which then requires another pixels array, then finally when you get your BufferedImage you have to have another pixel array to grab the data from it, in other words: nightmare.

    This can cause very slow speeds on some machines, and in no doubt it will cause slower performance on just about any machine. In 2004 someone submitted a bug report about it, 8 years later after being accepted nothing has been done.

    [Only registered and activated users can see links. ]

    I originally discovered these methods a few years ago, never really put them to use for anything, but now that I am writing an RDP I really needed them so I went further than just Windows support and asked around and got Windows, Linux, AND Mac 1.5/1.6/1.7 support. NOTICE: java.awt.Robot was not added until 1.5 so there is no reason to have anything earlier unless you want to write your own natives, then you probably aren't even interested in this..

    Yeah!

    Thanks to those who helping me get the dumps for the other platform methods.

    Danny (Clienthax) - Linux
    Daniel (Mister Maggot) - Mac 1.5 (or was it 1.6, idk)
    Avatar Realms (thanks to Danny for referring me) - Mac 1.6/1.7

    I also decided to include a more direct way of grabbing the mouse information, that doesn't involve the creation of a new object every single time...

    If anyone finds a platform that can't use the direct pixel grabbing method, please let me know and send me your dump.

    NOTE: I did leave the debugging stuff in there, so if you don't want it I suggest you just remove it. But please if it doesn't work post your results!

    And as always, have fun.

    Code:
    
    import java.awt.*;
    import java.awt.peer.*;
    import sun.awt.*;
    import java.lang.reflect.*;
    import java.net.*;
    import java.io.*;
    import java.util.*;
    
    public final class DirectRobot
    {
    	public DirectRobot() throws AWTException
    	{
    		this(null);
    	}
    
    	public DirectRobot(GraphicsDevice device) throws AWTException
    	{
    		if (device == null)
    			device = GraphicsEnvironment.getLocalGraphicsEnvironment().getDefaultScreenDevice();
    
    		this.device = device;
    		Toolkit toolkit = Toolkit.getDefaultToolkit();
    		peer = ((ComponentFactory) toolkit).createRobot(null, device);
    		Class<?> peerClass = peer.getClass();
    		Method method = null;
    		int methodType = -1;
    		Object methodParam = null;
    		try
    		{
    			method = peerClass.getDeclaredMethod("getRGBPixels", new Class<?>[] { Integer.TYPE, Integer.TYPE, Integer.TYPE, Integer.TYPE, int[].class });
    			methodType = 0;
    		}
    		catch (Exception ex) { }
    		if (methodType < 0)
    			try
    			{
    				method = peerClass.getDeclaredMethod("getScreenPixels", new Class<?>[] { Rectangle.class, int[].class });
    				methodType = 1;
    			}
    			catch (Exception ex) { }
    
    		if (methodType < 0)
    			try
    			{
    				method = peerClass.getDeclaredMethod("getScreenPixels", new Class<?>[] { Integer.TYPE, Rectangle.class, int[].class });
    				methodType = 2;
    				GraphicsDevice[] devices = GraphicsEnvironment.getLocalGraphicsEnvironment().
    getScreenDevices();
    				int count = devices.length;
    				for (int i = 0; i != count; ++i)
    					if (device.equals(devices[i]))
    					{
    						methodParam = Integer.valueOf(i);
    						break;
    					}
    
    			}
    			catch (Exception ex) { }
    
    		if (methodType < 0)
    			try
    			{
    				method = peerClass.getDeclaredMethod("getRGBPixelsImpl", new Class<?>[] { Class.forName("sun.awt.X11GraphicsConfig"), Integer.TYPE, Integer.TYPE, Integer.TYPE, Integer.TYPE, int[].class });
    				methodType = 3;
    				Field field = peerClass.getDeclaredField("xgc");
    				try
    				{
    					field.setAccessible(true);
    					methodParam = field.get(peer);
    				}
    				finally
    				{
    					field.setAccessible(false);
    				}
    			}
    			catch (Exception ex) { }
    
    		if (methodType >= 0 && method != null && (methodType <= 1 || methodParam != null))
    		{
    			getRGBPixelsMethod = method;
    			getRGBPixelsMethodType = methodType;
    			getRGBPixelsMethodParam = methodParam;
    		}
    		else
    		{
    			System.out.println("WARNING: Failed to acquire direct method for grabbing pixels, please post this on the main thread!");
    			System.out.println();
    			System.out.println(peer.getClass().getName());
    			System.out.println();
    			try
    			{
    				Method[] methods = peer.getClass().getDeclaredMethods();
    				for (Method method1 : methods)
    					System.out.println(method1);
    
    			}
    			catch (Exception ex) { }
    			System.out.println();
    		}
    	}
    
    	public static GraphicsDevice getMouseInfo(Point point)
    	{
    		if (!hasMouseInfoPeer)
    		{
    			hasMouseInfoPeer = true;
    			try
    			{
    				Toolkit toolkit = Toolkit.getDefaultToolkit();
    				Method method = toolkit.getClass().getDeclaredMethod("getMouseInfoPeer", new Class<?>[0]);
    				try
    				{
    					method.setAccessible(true);
    					mouseInfoPeer = (MouseInfoPeer) method.invoke(toolkit, new Object[0]);
    				}
    				finally
    				{
    					method.setAccessible(false);
    				}
    			}
    			catch (Exception ex) { }
    		}
    		if (mouseInfoPeer != null)
    		{
    			int device = mouseInfoPeer.fillPointWithCoords(point != null ? point:new Point());
    			GraphicsDevice[] devices = GraphicsEnvironment.getLocalGraphicsEnvironment().
    getScreenDevices();
    			return devices[device];
    		}
    		PointerInfo info = MouseInfo.getPointerInfo();
    		if (point != null)
    		{
    			Point location = info.getLocation();
    			point.x = location.x;
    			point.y = location.y;
    		}
    		return info.getDevice();
    	}
    
    	public static int getNumberOfMouseButtons()
    	{
    		return MouseInfo.getNumberOfButtons();
    	}
    
    	public static GraphicsDevice getDefaultScreenDevice()
    	{
    		return GraphicsEnvironment.getLocalGraphicsEnvironment().getDefaultScreenDevice();
    	}
    
    	public static GraphicsDevice getScreenDevice()
    	{
    		return getMouseInfo(null);
    	}
    
    	public void mouseMove(int x, int y)
    	{
    		peer.mouseMove(x, y);
    	}
    
    	public void mousePress(int buttons)
    	{
    		peer.mousePress(buttons);
    	}
    
    	public void mouseRelease(int buttons)
    	{
    		peer.mouseRelease(buttons);
    	}
    
    	public void mouseWheel(int wheelAmt)
    	{
    		peer.mouseWheel(wheelAmt);
    	}
    
    	public void keyPress(int keycode)
    	{
    		peer.keyPress(keycode);
    	}
    
    	public void keyRelease(int keycode)
    	{
    		peer.keyRelease(keycode);
    	}
    
    	public int getRGBPixel(int x, int y)
    	{
    		return peer.getRGBPixel(x, y);
    	}
    
    	public int[] getRGBPixels(Rectangle bounds)
    	{
    		return peer.getRGBPixels(bounds);
    	}
    
    	public boolean getRGBPixels(int x, int y, int width, int height, int[] pixels)
    	{
    		if (getRGBPixelsMethod != null)
    			try
    			{
    				if (getRGBPixelsMethodType == 0)
    					getRGBPixelsMethod.invoke(peer, new Object[] { Integer.valueOf(x), Integer.valueOf(y), Integer.valueOf(width), Integer.valueOf(height), pixels });
    				else if (getRGBPixelsMethodType == 1)
    					getRGBPixelsMethod.invoke(peer, new Object[] { new Rectangle(x, y, width, height), pixels });
    				else if (getRGBPixelsMethodType == 2)
    					getRGBPixelsMethod.invoke(peer, new Object[] { getRGBPixelsMethodParam, new Rectangle(x, y, width, height), pixels });
    				else
    					getRGBPixelsMethod.invoke(peer, new Object[] { getRGBPixelsMethodParam, Integer.valueOf(x), Integer.valueOf(y), Integer.valueOf(width), Integer.valueOf(height), pixels });
    
    				return true;
    			}
    			catch (Exception ex) { }
    
    		int[] tmp = getRGBPixels(new Rectangle(x, y, width, height));
    		System.arraycopy(tmp, 0, pixels, 0, width * height);
    		return false;
    	}
    
    	public void dispose()
    	{
    		getRGBPixelsMethodParam = null;
    		Method method = getRGBPixelsMethod;
    		if (method != null)
    		{
    			getRGBPixelsMethod = null;
    			try
    			{
    				method.setAccessible(false);
    			}
    			catch (Exception ex) { }
    		}
    		//Using reflection now because of some peers not having ANY support at all (1.5)
    		try
    		{
    			peer.getClass().getDeclaredMethod("dispose", new Class<?>[0]).invoke(peer, new Class<?>[0]);
    		}
    		catch (Exception ex) { }
    	}
    
    	protected void finalize() throws Throwable
    	{
    		try
    		{
    			dispose();
    		}
    		finally
    		{
    			super.finalize();
    		}
    	}
    
    	private Object getRGBPixelsMethodParam;
    	private int getRGBPixelsMethodType;
    	public final GraphicsDevice device;
    	private Method getRGBPixelsMethod;
    	private final RobotPeer peer;
    	private static boolean hasMouseInfoPeer;
    	private static MouseInfoPeer mouseInfoPeer;
    }
    Still need a BufferedImage?

    Use this, trust me there is no copying involved here, you change a pixel in the array and the image changes too, try it! (this is what Jagex uses for their custom pixel rendering)

    Code:
    ColorModel model = new DirectColorModel(32, 0xff0000, 0xff00, 0xff, 0xff000000);
    BufferedImage image = new BufferedImage(model, Raster.createWritableRaster(model.createCompatibleSampleModel(width, height), new DataBufferInt(pixels, width * height), null), false, new Hashtable<Object, Object>());
    Remember though, if your pixels are JUST RGB and not ARGB (alpha + rgb), then you need to remove "0xff000000" from the DirectColorModel arguments, just remove it. But the Robot should return ARGB, if not let me know. But chances are if you are using this for something else and you do not know what ARGB means, then you have RGB.
    Reply With Quote  
     

  2. Thankful users:


  3. #2  
    Banned
    Join Date
    Jan 2012
    Age
    20
    Posts
    552
    Thanks given
    53
    Thanks received
    37
    Rep Power
    0
    is this like a rat?
    Reply With Quote  
     

  4. #3  
    Registered Member configuration_'s Avatar
    Join Date
    Feb 2012
    Posts
    47
    Thanks given
    0
    Thanks received
    12
    Rep Power
    42
    could be used for other bad stoffs
    Shark.
    Reply With Quote  
     

  5. #4  
    Registered Member
    Killer 99's Avatar
    Join Date
    Dec 2007
    Posts
    1,484
    Thanks given
    171
    Thanks received
    508
    Rep Power
    414
    Quote Originally Posted by Tha King View Post
    is this like a rat?
    no.. this is just used for faster ("direct") access to the java.awt.Robot class......


    Quote Originally Posted by configuration_ View Post
    could be used for other bad stoffs
    so could java.awt.Robot???
    Reply With Quote  
     

  6. #5  
    Registered Member
    Killer 99's Avatar
    Join Date
    Dec 2007
    Posts
    1,484
    Thanks given
    171
    Thanks received
    508
    Rep Power
    414
    Fixed the post, for some reason the code tag messed up or i messed up
    Reply With Quote  
     

  7. #6  
    Janemba!!!!

    Women's Avatar
    Join Date
    Mar 2010
    Posts
    1,653
    Thanks given
    395
    Thanks received
    681
    Rep Power
    441
    Code:
    import java.awt.*;
    import java.awt.peer.*;
    import sun.awt.*;
    import java.lang.reflect.*;
    import java.net.*;
    import java.io.*;
    import java.util.*;
    
    public final class DirectRobot
    {
    	public DirectRobot() throws AWTException
    	{
    		this(null);
    	}
    
    	public DirectRobot(GraphicsDevice device) throws AWTException
    	{
    		if (device == null)
    			device = GraphicsEnvironment.getLocalGraphicsEnvironment().getDefaultScreenDevice();
    
    		this.device = device;
    		Toolkit toolkit = Toolkit.getDefaultToolkit();
    		peer = ((ComponentFactory) toolkit).createRobot(null, device);
    		Class<?> peerClass = peer.getClass();
    		Method method = null;
    		int methodType = -1;
    		Object methodParam = null;
    		try
    		{
    			method = peerClass.getDeclaredMethod("getRGBPixels", new Class<?>[] { Integer.TYPE, Integer.TYPE, Integer.TYPE, Integer.TYPE, int[].class });
    		
    import java.awt.*;
    import java.awt.peer.*;
    import sun.awt.*;
    import java.lang.reflect.*;
    import java.net.*;
    import java.io.*;
    import java.util.*;
    
    public final class DirectRobot
    {
    	public DirectRobot() throws AWTException
    	{
    		this(null);
    	}
    still a bit messed up at the top
    Spoiler for too big pal:
    Reply With Quote  
     

  8. #7  
    Registered Member
    Killer 99's Avatar
    Join Date
    Dec 2007
    Posts
    1,484
    Thanks given
    171
    Thanks received
    508
    Rep Power
    414
    Sorry, either chrome was failing or my mouse is failing (i think i need a new mouse already tho)
    Reply With Quote  
     

  9. #8  
    PokeNation! Xaves's Avatar
    Join Date
    Dec 2007
    Age
    26
    Posts
    3,478
    Thanks given
    356
    Thanks received
    780
    Rep Power
    621
    Hmm, I'll test this later today


    [Only registered and activated users can see links. ]

    [Only registered and activated users can see links. ]
    Reply With Quote  
     

  10. #9  
    SERGEANT OF THE MASTER SERGEANTS MOST IMPORTANT PERSON OF EXTREME SERGEANTS TO THE MAX!


    S Quare Quxx's Avatar
    Join Date
    Jun 2007
    Posts
    8,668
    Thanks given
    1,825
    Thanks received
    4,464
    Rep Power
    5000
    WARNING: Failed to acquire direct method for grabbing pixels, please post this on the main thread!

    sun.awt.X11.XRobotPeer

    private static synchronized native void sun.awt.X11.XRobotPeer.setup()
    public void sun.awt.X11.XRobotPeer.mousePress(int)
    public int[] sun.awt.X11.XRobotPeer.getRGBPixels(java.awt.Recta ngle)
    private static synchronized native void sun.awt.X11.XRobotPeer.getRGBPixelsImpl(sun.awt.X1 1GraphicsConfig,int,int,int,int,int[])
    public void sun.awt.X11.XRobotPeer.mouseMove(int,int)
    public void sun.awt.X11.XRobotPeer.mouseRelease(int)
    public void sun.awt.X11.XRobotPeer.mouseWheel(int)
    public void sun.awt.X11.XRobotPeer.keyPress(int)
    public void sun.awt.X11.XRobotPeer.keyRelease(int)
    public int sun.awt.X11.XRobotPeer.getRGBPixel(int,int)
    public void sun.awt.X11.XRobotPeer.dispose()
    private static synchronized native void sun.awt.X11.XRobotPeer.mouseMoveImpl(sun.awt.X11Gr aphicsConfig,int,int)
    private static synchronized native void sun.awt.X11.XRobotPeer.mousePressImpl(int)
    private static synchronized native void sun.awt.X11.XRobotPeer.mouseReleaseImpl(int)
    private static synchronized native void sun.awt.X11.XRobotPeer.mouseWheelImpl(int)
    private static synchronized native void sun.awt.X11.XRobotPeer.keyPressImpl(int)
    private static synchronized native void sun.awt.X11.XRobotPeer.keyReleaseImpl(int)

    arch linux 3.2.6 x86_64 xorg & openbox

    default screen device doesn't work, nor does this

    Code:
    GraphicsDevice[] devices = GraphicsEnvironment.getLocalGraphicsEnvironment().getScreenDevices();
    for(GraphicsDevice device : devices) {
    	try {
    		robot = new DirectRobot(device);
    	} catch (AWTException e) {
    		continue;
    	}
    }



    Reply With Quote  
     

  11. #10  
    Registered Member
    Killer 99's Avatar
    Join Date
    Dec 2007
    Posts
    1,484
    Thanks given
    171
    Thanks received
    508
    Rep Power
    414
    Quote Originally Posted by S Quare Quxx View Post
    WARNING: Failed to acquire direct method for grabbing pixels, please post this on the main thread!

    sun.awt.X11.XRobotPeer

    private static synchronized native void sun.awt.X11.XRobotPeer.setup()
    public void sun.awt.X11.XRobotPeer.mousePress(int)
    public int[] sun.awt.X11.XRobotPeer.getRGBPixels(java.awt.Recta ngle)
    private static synchronized native void sun.awt.X11.XRobotPeer.getRGBPixelsImpl(sun.awt.X1 1GraphicsConfig,int,int,int,int,int[])
    public void sun.awt.X11.XRobotPeer.mouseMove(int,int)
    public void sun.awt.X11.XRobotPeer.mouseRelease(int)
    public void sun.awt.X11.XRobotPeer.mouseWheel(int)
    public void sun.awt.X11.XRobotPeer.keyPress(int)
    public void sun.awt.X11.XRobotPeer.keyRelease(int)
    public int sun.awt.X11.XRobotPeer.getRGBPixel(int,int)
    public void sun.awt.X11.XRobotPeer.dispose()
    private static synchronized native void sun.awt.X11.XRobotPeer.mouseMoveImpl(sun.awt.X11Gr aphicsConfig,int,int)
    private static synchronized native void sun.awt.X11.XRobotPeer.mousePressImpl(int)
    private static synchronized native void sun.awt.X11.XRobotPeer.mouseReleaseImpl(int)
    private static synchronized native void sun.awt.X11.XRobotPeer.mouseWheelImpl(int)
    private static synchronized native void sun.awt.X11.XRobotPeer.keyPressImpl(int)
    private static synchronized native void sun.awt.X11.XRobotPeer.keyReleaseImpl(int)

    arch linux 3.2.6 x86_64 xorg & openbox

    default screen device doesn't work, nor does this

    Code:
    GraphicsDevice[] devices = GraphicsEnvironment.getLocalGraphicsEnvironment().getScreenDevices();
    for(GraphicsDevice device : devices) {
    	try {
    		robot = new DirectRobot(device);
    	} catch (AWTException e) {
    		continue;
    	}
    }
    it's fixed, it was just a little mistake sorry
    Reply With Quote  
     

Page 1 of 3 123 LastLast

Thread Information
Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. Java Robot + Gui HELP NEEDED
    By Bridget7298 in forum Help
    Replies: 5
    Last Post: 07-02-2011, 11:09 PM
  2. Full Screen Recording
    By I Hax in forum PC
    Replies: 7
    Last Post: 05-08-2010, 05:25 PM
  3. using java.awt.Robot
    By Discardedx2 in forum Help
    Replies: 2
    Last Post: 04-14-2010, 09:51 PM
Posting Permissions
  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •