-

Big Data Machine Learning with H2O


a years ago SriSatish Ambati gave a talk at the Chicago Big Data meetup about how to use H2O to do machine learning on big data.
SriSatish is founder and CEO of a startup called 0xdata (pronounced "hexadata").
0xdata developed H2O, an open-source machine learning tool. (Github.)
H2O has a number of powerful machine learning algorithms that it can run. What makes H2O different from R and other programs is that H2O has really good performance and can handle huge amounts of data. It splits up the algorithm and data into little chunks and processes them in parallel over many machines in a cluster.
Netflix is using H2O to do some of their machine learning. Netflix is one of the pioneers of big data machine learning, and I look forward to hear more about how they're using H2O.
You can run H2O against data stored in the Hadoop Distributed File System (HDFS). You can also upload data directly to the H2O cluster.
H2O is written in Java, but because it works at the data / analysis layer, it should play nice with most systems.
H2O currently supports the following algorithms:
  • Random Forests
  • Generalized Linear Modeling (GLM)
  • Logistic regression
  • K-Means
SriSatish said the team is working to add more algorithms. In particular, SriSatish told us, a key area of focus is developing a set of statistics and math algorithms for unbalanced datasets for use in fraud detection.
There is also an H2O R package, so R users can run distributed H2O algorithms from the R command line.
below is a syntax of R command i run to initialise h2o in R

#Start the H2O server locally.
> localH2o = h2o.init(ip = "127.0.0.1", port = 54321) 
> filepath =file.choose()

> test_data <- h2o.uploadFile(file.choose(), destination_frame = "",     parse = T, header = T, sep = ",", 
    na.strings = c("unknown"), progressBar = FALSE, 
  parse_type = "csv")

The algorithms can also be run via a self-hosted web interface.
SriSatish said that the company's goal is to enable developers to use H2O as a real-time machine learning service. To enable a smartphone app or web app to learn continually as it obtained more data.
"Our real goal is to make algorithms embeddable in your application."—SriSatish Ambati, CEO of 0xdata;/.l,

Comments

Post a Comment

Popular posts from this blog

Driving Visual Analysis with Automobile Data (R)

-
Acquiring automobile fuel efficiency data Every  data science project starts with data and this chapter is no different. For this recipe, we will dive into a dataset that contains fuel efficiency performance metrics, measured in  miles per gallon  ( MPG ) over time, for  most makes and models of automobiles available in the U.S. since 1984. This data is courtesy of the U.S. Department of Energy and the US Environmental Protection Agency. In addition to fuel efficiency data, the dataset also contains several features and attributes of the automobiles listed, thereby providing the opportunity to summarize and group data to determine which groups tend to have better fuel efficiency historically and how this has changed over the years. The latest version of the dataset is  available at  http://www.fueleconomy.gov/feg/epadata/vehicles.csv.zip , and information about  the variables in the dataset can be found at  http://www.fueleconomy.gov/feg/ws/index.shtml#vehicle . The data was l
Read more

Evaluating Classification Model Performance

-
Image
Evaluating Model Performance When only the wealthy could afford education, tests and exams did not evaluate students' potential. Instead, teachers were judged for parents who wanted to know whether their children had learned enough to justify the instructors' wages. Obviously, this has changed over the years. Now, such evaluations are used to distinguish between high- and low-achieving students, filtering them into careers and other opportunities. Given the significance of this process, a great deal of effort is invested in developing accurate student assessments. Fair assessments have a large number of questions that cover a wide breadth of topics and reward true knowledge over lucky guesses. They also require students to think about problems they have never faced before. Correct responses therefore indicate that students can generalize their knowledge more broadly. The process of evaluating machine learning algorithms is very similar to the process of evaluating stu
Read more

Practical Employment project with R

-
Image
Visually Exploring Employment Data (R) In this Articule, we will cover: Preparing for analysis Importing employment data into R Exploring the employment data Obtaining and merging additional data Adding geographical information Extracting state- and county-level wage and employment information Visualizing geographical distributions of pay Exploring where the jobs are, by industry Animating maps for a geospatial time series Benchmarking performance for some common tasks Preparing for analysis This  recipe will prepare the groundwork with the tools you need to complete this project. If you do not have R installed on your computer, please see the instructions in  Chapter 1 ,  Preparing Your Data Science Environment . Getting ready You need a computer with R installed and an Internet connection. How to do it… The following steps will get you prepared for the remainder of this chapter by downloading the dataset fro
Read more