Neo4j Inc. targets artificial intelligence and machine learning applications with a new version of its graph database announced today.
Graphics engines differ from conventional relational and NoSQL databases in that they document the connections between data items. This allows organizations to map relationships that would be impractical or impossible to represent in other database engines, such as those between all of the characters in “Game of Thrones” (pictured).
Graphical databases are increasingly used in AI scenarios because of this unique capability, said Philip Rathle, vice president of products at Neo4j. “The problem with NoSQL databases is that they provide you with data but not the connections,” he said. “People are now looking to make connections to make better decisions based on context. For example, AI-driven voice response systems are most effective when they understand the context of a command.
Graphics engines are also well suited to scenarios where decisions made by an algorithm need to be explained, Rathle said. For example, if a customer of a bank disputes a denied loan application, “the bank and regulators will want to know why the machine made this decision,” he said. “Many machine learning and deep learning processes [based upon conventional databases] are so complex that it can take weeks to figure out.
Version 3.5 adds full-text indexing to enable text-intensive applications such as metadata management and BOM processing. The feature, which is based on the open source Lucene engine, “is one of the most requested features that we have had for years,” said Rathle.
Support for native indexes has been extended to include spatial, temporal, and Boolean values as well as composite indexing. The result is up to five times the performance improvement, the company said. “It turned out that 80% of Neo4J’s time was spent indexing when large data was ingested,” said Rathle. Native indexes are now also used for sorting operations for queries written in the Cypher language of Neo4j.
The second most common user request was for improving the handling of large write transactions. A new memory subsystem has therefore been added for this purpose. The so-called non-heap transaction subsystem, along with clustering, uses native memory more efficiently to propagate large writes throughout the cluster. Non-heap memory management bypasses the garbage cleaning process in Java which can create processing pauses. “We implemented specific low-level memory management rather than relying on Java,” said Rathle.
Other new features include a driver for the increasingly popular Go programming language developed by Google LLC and graphical algorithms for unsupervised learning methods such as Random Walks, Custom PageRank, Similarities, DeepGL and DeepWalk.