A platform integrating reinforcement learning algorithms in the General Game Playing setting using RL-glue
This project was created with the intention to test different types of reinforcement learning algorithms in different types of games. This is more difficult than it sounds, since while there are tools and interfaces that allow for a general game description using GDL, we lack a general interface to use generic reinforcement learning algorithms for game playing.
In order to integrate reinforcement learning algorithms to play general games it is necessary to adjust the rewards that agents receive in such a way that the algorithms learn (at the same time) the rules and the strategies of the same.
In this context this project combines different tools and interfaces towards this goal.
In this way, we have integrated RL-Glue with the GGP-Server, which loads the games through the Game Description Language.
RL-Glue tool, despite its potential, does not (yet) have a complete library of reinforcement learning algorithms. Therefore, in order to work with various algorithms we also integrated the van Hasselt's algorithm library (http://homepages.cwi.nl/~hasselt/) in RL-Glue, so that they can apply them to any game.
RL-GGP has used the Java version of all tools (or migrated code to Java), so it is now entirely in Java, and is therefore platform-independent. The scheme of the system is shown in the following figure:
For any other use or any commercial use *do* contact the authors.
Dozens of GDL games from Dresden's GGP server
If you cannot find more GDL games in the GGP servers, you can download a few here.
First step, install RL-Glue Java codec:
The codec can be downloaded either as a .tar.gz or can be checked out of the subversion repository where it is hosted.
The tarball distribution can be found here:
http://code.google.com/p/rl-glue-ext/wiki/Java
You can start by checking the subversion code (subversion can be downloaded from http://subversion.apache.org/) execute:
svn checkout http://rl-glue-ext.googlecode.com/svn/trunk/projects/codecs/Java Java-Codec
Technically, all you really need is the JAR archive of the codec:
svn export http://rl-glue-ext.googlecode.com/svn/trunk/projects/codecs/Java/products/JavaRLGlueCodec.jar
This codec can either be installed into a Java extensions directory, or it can be used uninstalled in ``free-float'' mode by specifying it in the Java classpath, when compiling and running Java classes.
The advantage of the installed mode is that your system will always know where the RL-Glue classes are, so the code required to compile and run the applications is much cleaner. The downside of installing them is that Java will always find the extension classes first, meaning that you can not easily move between different versions of the classes just by specifying a different classpath.
Below there are two examples of how you would work with the code, and then you can choose for yourself whether to install or not. The rest of this manual's instructions will all be written as if you have the classes installed, for clarity.
//Not installed (free-float) mode >$ javac -cp path/to/codecs/Java/products/JavaRLGlueCodec.jar MyAgent.java >$ java -cp path/to/codecs/Java/products/JavaRLGlueCodec.jar:. MyAgent // Installed mode >$ javac MyAgent.java >$ java MyAgent
To install the RL-Glue Java Extension, do the following:
//This location will depend on whether you have the developer distribution // or the user distribution // User Distribution: do this >$ cd path/to/downloaded/codec/Java/ // Developer Distribution: do this >$ cd path/to/downloaded/codec/Java/products //Both distributions are installed in the same way >$ java -jar JavaRLGlueCodec.jar --installThis will provide you with a numbered list, prompting you to choose a Java extension folder to install to that is appropriate for your system.
//Test it by typing: >$ java org.rlcommunity.rlglue.codec.RLGlueCore --version
Second step, download and extract RL-GGP:
You can download de latest version of RL-GGP here.
Once downloaded, extract the required directory.
Third step, create a gamer:
The user would only have to create a player with the algorithms developed by the user or with the algorithms which are provided by default, creating a *.jar of the class "main" of Jocular in the .\lib\jocular-0.2\jocular\src\java\stanfordlogic\jocular, where RL-GGP has been extracted. It is recommended that these players use the Eclipse tool, although in principle you can use any programming environment for implementing any change.
After you create the *. jar, you just have to use a console and call as it follows:
>$ java -jar jocular_remote.jar --port=4001In this case the player (jocular_remote.jar) would be listening on port 4001. Finally, in order to run the player you must create a file "config_agent.cfg" in the same directory where the player is (you can find an example of this file in the directory where the RL-GGP has been extraceted) . This file specifies the algorithm to be executed. You can download an example of the default jocular gamer of RL-GGP here.
Fourth step, execute GGP-Server and load game GDL:
If the user wants to run the server default GGP-Server simply type the directory .\ GGP-Server-exec\start_ggp_server.bat (where the RL-GGP implementation has been extracted) for Windows. If you use Linux, move to this directory in the command line and execute the following:
>$ java -jar ggp_server.jarThen, we only need to specify the file with the GDL game (for example Tic Tac Toe or connect 4) and the connection parameters of each remote player which is listening, by selecting the host, port and REMOTE type, or if you use the default players given by GGP's own server.
For more information about the use, installation, requirements or code of RL-GGP, you can download the project specifications RL-GGP (in spanish) here.
A summary (in English) of the integration of van Hasselt's RL library into RL-glue can be founde here.
A summary (in English) of the integration of RL-glue and GGP via Jocular can be found here.