Java, the Steam controller and me

Attila Mihaly BalazsJanuary 8th, 2014Last Updated: January 8th, 2014

1 130 5 minutes read

Do you ever wondered if you could use existing stuff for something new? I saw some footage of the so-called ‘Steam controller’ (SC from now on) and looked at my game pad. Asking myself if it would be possible to use it in a steamy-like way, I’ve found some Java libraries and created a project that I’d like to share with you today.

Of course, there have been a lot of input devices (and especially game controllers) long before the SC’s release, but it has one new property which makes it special.

It has two touchpads, which can emulate a mouse’s or keyboard’s input in order to be able to play (practically nearly) every game. As some early videos show, even a mouse-intensive game like the puzzle game ‘Portal’ seems to be playable by using this compatibility mode.
As a gamer enthusiast and Java programmer, what could I do with something like this (a XBOX controller which I already got) in order to come close to that?

A small tool named ‘StrangeCtrl’ saw the world’s bright light. Talking to the controller needs some JNI (because there is no USB subsystem in the JVM for example), but the rest is written in pure Java. It sits in the system tray and is configured manually per configuration file, although one could build a GUI, too.

Its dependencies are ‘net.java.jinput.JInput’ in version 2.0.5 (still working for Windows 8.1) and a little helper I wrote (‘com.xafero.SuperLoader’ v0.1). Now I’ll explain the steps taken on the way.

First step: How do we get Java to talk to my controller?

Luckily, the BSD-licensed JInput project does exactly this. It connects to Microsoft’s XInput interface for example and fills some Java data structures with the native data it gets. Linux and Mac OS X are covered, too, don’t worry.

So I plugged in my game pad (one XBOX-compatible controller) and the way seemed to be clear:

get the controllers
get their input events
and convert them to virtual events for keyboard and mouse.

The library’s native components for the big three OS are delivered in Java archives (at least per Maven). But as you might already know, java.lang.System only loads files directly available on the file system.

Second step: So how do we get around this annoying limitation?

After a quick search, I’ve found wcmatthysen’s ‘mx-native-loader’ which seemed useful as it claims to extract JAR’s and load the native stuff. But it didn’t work, because the libraries of JInput are packed into several ‘jinput-platform-***.jar’ files instead of one big chunk under META-INF/lib as this loader suggests.

So the new helper library called ‘SuperLoader’ works around these circumstances:

Create a temporary directory for all nasty native libraries, e.g. with the help of the system property ‘java.io.tmpdir’. It could also be specified by the user directly as it doesn’t really matter where it is.
Get all nasty libraries out of the JARs already loaded; iterate over all class path’s URLs and extract them or exclude most of them by using a filter.
Extend the existing library path; one thing the other library didn’t do and it’s very annoying to do it manually, so the system property ‘java.library.path’ should be extended.
Force the JVM to renew the system paths; one can do it by resetting the field ‘sys_paths’ of the system class loader to null. This forces the System class to really appreciate the new circumstances the next time you request a library.

Now the application pre-loads all native libraries into a temporary folder and when JInput is asked to give a list of the controllers, for example, it doesn’t have to be changed for using JAR files. It simply is able to use System.loadLibrary as anyone would do.

Third step: What’s possible to simulate?

We’ve finally got to read the game pad’s events, so what can we do with it? With AWT’s Robot class, it’s possible since the early Java days to simulate a key press or mouse movements and such. Although the robot requires one to specify the desktop it should work on, it works just as good on multi-monitor systems. The only difference is the offset of all events it generates – an aspect especially important if one wants to click on specific regions of the PC’s screen.

The implemented commands so far are:

MouseMoveCmd – moves the mouse by some amount horizontally or vertically
MouseClickCmd – clicks the given mouse button at the current screen position
KeyComboCmd – presses some keys and releases them in the reversed order

To allow some bit of extensibility, there is an interface which accepts the robot to generate virtual events, the current graphics device and the value given by JInput:

public interface ICommand {
    void execute(Robot rbt, GraphicsDevice dev, float value);
}

Its abstract implementation ‘AbstractCmd’ provides one constructor accepting one string. As a first step of processing, the raw string coming from the configuration file is splitted by an empty space into a string array.

Fourth step: Which configuration format can we use?

There are a lot of trendy formats out there, like YAML, JSON, … But Java already provides us with a simple way to achieve this. So the configuration file is parsed with the XML variant of the Java properties mechanism. To build the actual map out of strings connected with their commands, the class ‘com.xafero.strangectrl.cmd.ConfigUtils’

loads the configuration,
iterates through all entries,
- searches a command by each entry’s value,
- loads each command by instantiating it with the textual arguments,
- puts the result of key (controller button) and value (associated command) into a new map,
and produces the actual map used to transform incoming events.

Fifth step: The actual work…

The helper class ‘ControllerPoller’ is a periodically executed TimerTask who is responsible for collecting new JInput events from an arbitrary amount of controllers and inform the caller about every new stuff:

public void run() {
for (Controller controller : controllers) {
if (!controller.poll()) continue;
EventQueue queue = controller.getEventQueue();
Event event = new Event();
while (queue.getNextEvent(event))
callback.onNewEvent(this, controller, event);
}
}

The caller (in this case the so-called ‘App’ living in the system tray) just implements the callback interface and gets all information for free whenever some input occurs:

public static interface IControllerCallback {
void onNewEvent(ControllerPoller p, Controller c, Event e);
}

Left to the ‘App’ is the search for associated commands to the incoming game pad’s events and their execution with the correct parameters. Now we could use it for controlling some game, perhaps an old one like Prince of Persia or something otherwise unplayable with a game pad. But let’s step aside…

Example aside from games: How to configure it for people with constrained movement?

To show just another possible field of application, let’s configure it for someone who can’t press two keys at the same time. An example application should here be a web browser. In the configuration file, there are the following settings:

<!-- Button A means now left mouse click -->
<entry key="Button 0">mouseClick 1</entry>
<!-- Button B will open a new tab -->
<entry key="Button 1">keyCombo CONTROL T</entry>
<!-- Button X will close an existing tab -->
<entry key="Button 2">keyCombo CONTROL W</entry>

The browser in this example doesn’t have to know the game controller, because the operating system will produce new virtual input events, and it will operate as demanded. By using Java and being FOSS, the tool is also customizable and easily understandable in every way (in contrast to some C/C++ code which would be otherwise necessary to emulate input devices).