It has been my dream since I was a kid and I saw the Matthew Broderick movie War Games in a theater in 1983 to build an automated trading system using AI. This year I have begun to live that dream. I call this project jVest.
Curious how I built it? Here is a description of what I created:
jVest is a real-time high transaction volume market analysis and trading system implementing machine learning for predictive analytics. It has an HTML 5 UI using WebSockets to continuously stream real-time information to the browser. The UI itself is largely built using Angular 2 with Typescript. It uses JMS to distribute the incoming third-party data messages after converting them to POJOs to subscribers. Using a listener, it persists select data in a RDBMS using JPA. Using another listener, it is streamed to business logic which then streams its transformed output to the web UI via WebSockets. It also uses the Observer pattern to flow the data through many consumers and producers, and injection (CDI) to simplify the complex interactions and provide a highly extensible design.
It uses REST for user interactions with the server and WebSockets for real-time streaming. To handle conversion of POJOs to/from XML, it leverages JAXB to marshal and leverages the marshalling capabilities of Resteasy in the REST layer. All transportable POJOs implement a Marshallable interface providing common default methods to do JAXB mashalling to/from XML and JSON, which is used for WebSockets as well as conversion of XML to POJOs from upstream data providers.
For machine learning, it uses JServe to integrate with R over TCP/IP, which permits high scalability through distributed R nodes. R then handles the predictive algorithms such as linear regression and KNN. Users can query via the web UI to analyze market data, as well as create monitors that regularly update the predictions in the web UI via WebSockets.
As you can see, it is not only a fun project, but has brushed up skills by using the latest updates to the Java EE stack. On top of that, I learned completely new things, such as Angular 2 with Typescript which is becoming a very popular UI solution that encourages creating highly re-usable UI components; and R, one of the leading languages for doing machine learnings with a large plethora of readily available statistical analysis packages.
Using R to do machine learning is not only cutting edge in terms of technology, it is a rapidly growing area due to the proliferation of large quantities of data in computers, faster computers, and cheaper storage.
Limits to computer speed and storage were the reasons I set aside pursuing AI in the 80s after my initial dabbling with it. It is exciting to be able to pick up that dream today now that hardware can now do things barely imaginable in the 80s.
Oct 28, 2016 – Added clustering via WebSockets
I added the ability to cluster servers using WebSockets. This overcomes a limitation. The upstream data provider permits only one live real-time stream per account, though all the nodes can use the upstream provider’s synchronous API (think “REST”). To overcome the stream limitation, other nodes can now connect via WebSockets to receive the real-time streaming data. Because each instance of the application can act as both a provider and consumer, this allows for theoretically unlimited scaling of the business process and web/UI tier through a hierarchical topology. Because all streaming services are initiated through REST or scheduling, this is 100% dynamically configurable at run-time, both from a user-driven perspective and, down the road, perhaps for automated discovery, load balancing and high availability.
Technical description: this uses WebSockets for Java-to-Java in an EE web container. The provider endpoint picks up messages to send using @Observes, and the client side fires the messages it receives just as it would if they were received in its MDB if it was running the data collector that handles real-time streaming with the upstream provider. This demonstrates the powerful plug-ability and extensibility of the Observer pattern. It uses a WebSockets encoder and decoder to convert all messages to/from JSON for serialization. The en/decoders were easy to create since all message POJOs entering and exiting the endpoints internally support JAXB bi-directional marshalling of both XML and JSON accessible through a common interface.