Menu
Big Data Application Framework Gets Update, SQL Interface

Big Data Application Framework Gets Update, SQL Interface

Building on last month's release of Apache Hadoop 2.2, big data application platform specialist Concurrent today released a new version of Cascading, its big data application framework.

"I created Cascading in anger after having used MapReduce once in my life and vowing never to use it again." -- Chris Wensel, Founder and CTO of Concurrent.

Concurrent also announced the general availability of Cascading Lingual 1.0, an open source project that provides a comprehensive ANSI SQL interface.

Cascading is a stand-alone open source Java application framework designed as an alternative API to MapReduce. Cascading gives Java developers the capability to build big data applications on Hadoop using their existing skillset.

"I created Cascading in anger after having used MapReduce once in my life and vowing never to use it again," says Chris Wensel, creator of Cascading and founder and CTO of Concurrent.

The latest release, Cascading 2.5 adds support for Hadoop 2.2, including the new YARN architecture introduced in that version of Hadoop. Apache Hadoop YARN (Yet Another Resource Negotiator) serves as the Hadoop operating system, taking what was a single-use data platform for batch processing and evolving it into a multi-use platform that enables batch, interactive, online and stream processing.

YARN acts as the primary resource manager and mediator of access to data stored in Hadoop Distributed File System (HDFS), giving enterprises the capability to store data in a single place and then interact with it in multiple ways, simultaneously, with consistent levels of service.

Enterprises can now use Cascading to leverage Java, legacy SQL and predictive modeling investments for a single big data processing application.

Migration Path to Hadoop 2

Gary Nakamura, CEO of Concurrent, says that Cascading doesn't leverage YARN specifically, but does enable users to seamlessly migrate their applications to Hadoop 2 and take advantage of YARN. Domain specific languages (DSLs) like Scalding, Cascalog and PyCascading also seamlessly migrate to Hadoop 2. Similarly, Cascading will support Apache Tez when it takes its place in the Hadoop stack.

Concurrent has also added performance improvements for complex join operations and optimizations to dynamically partition and store processed data more efficiently on HDFS.

In addition to Cascading, Concurrent announced the immediate availability of Cascading Lingual 1.0, intended to help enterprises that have already invested heavily in business intelligence (BI) tools like Pentaho, Jaspersoft and Cognos-and the training to go with them-to quickly access their data on Hadoop. Lingual allows users to utilize their existing SQL skills and systems to create and run applications on Hadoop.

Concurrent's Wensel says Lingual empowers just about anyone familiar with SQL to instantly work with data stored on Hadoop using their JDBC-compliant BI or desktop tool of choice.

"Cascading is an important component to the big data application development ecosystem, and Lingual is another step forward in making it significantly easier to build big data apps," says Steve McPherson, group manager, Amazon Elastic MapReduce (EMR) at Amazon Web Services (AWS).

"Now, Amazon Elastic MapReduce customers can leverage Lingual to integrate disparate data stores on Amazon Web Services with services such as Amazon S3 and Amazon Redshift, and they can process the data and store it in Amazon EMR through one standard ANSI SQL statement," McPherson says. "This makes it easier for customers to query data with their favorite BI tool."

Thor Olavsrud covers IT Security, Big Data, Open Source, Microsoft Tools and Servers for CIO.com. Follow Thor on Twitter @ThorOlavsrud. Follow everything from CIO.com on Twitter @CIOonline, Facebook, Google + and LinkedIn.

Read more about applications in CIO's Applications Drilldown.


Follow Us

Join the New Zealand Reseller News newsletter!

Error: Please check your email address.

Tags applicationshadoopCascadingjavaLingualdata miningsoftwareConcurrentbig dataMapRSQLapache

Featured

Slideshows

Sizing up the NZ security spectrum - Where's the channel sweet spot?

Sizing up the NZ security spectrum - Where's the channel sweet spot?

From new extortion schemes, outside threats and rising cyber attacks, the art of securing the enterprise has seldom been so complex or challenging. With distance no longer a viable defence, Kiwi businesses are fighting to stay ahead of the security curve. In total, 28 per cent of local businesses faced a cyber attack last year, with the number in New Zealand set to rise in 2017. Yet amidst the sensationalism, media headlines and ongoing high profile breaches, confusion floods the channel, as partners seek strategic methods to combat rising sophistication from attackers. In sizing up the security spectrum, this Reseller News roundtable - in association with F5 Networks, Kaspersky Lab, Tech Data, Sophos and SonicWall - assessed where the channel sweet spot is within the New Zealand channel. Photos by Maria Stefina.

Sizing up the NZ security spectrum - Where's the channel sweet spot?
Kiwi channel comes together for another round of After Hours

Kiwi channel comes together for another round of After Hours

The channel came together for another round of After Hours, with a bumper crowd of distributors, vendors and partners descending on The Jefferson in Auckland. Photos by Maria Stefina.​

Kiwi channel comes together for another round of After Hours
Consegna comes to town with AWS cloud offerings launch in Auckland

Consegna comes to town with AWS cloud offerings launch in Auckland

Emerging start-up Consegna has officially launched its cloud offerings in the New Zealand market, through a kick-off event held at Seafarers Building in Auckland.​ Founded in June 2016, the Auckland-based business is backed by AWS and supported by a global team of cloud specialists, leveraging global managed services partnerships with Rackspace locally.

Consegna comes to town with AWS cloud offerings launch in Auckland
Show Comments