ODPS ali cloud, big data into power mode

which is so far the most important product of cloud, ali “nuclear weapons grade” big data platform ODPS formal business recently. Simply by ODPS online services, small companies spend hundreds of yuan can analyze huge amounts of data. ODPS can handle 100 pb data within 6 hours, equivalent to 100 million high-definition movie, associated with large data engine, after baidu released data platform products FusionInsight last year, huawei, and tianhe 2 super computer the ability to open, you can see China is experiencing a wave of large data capacity open trend.

ODPS is what? Open big data engine

ODPS full Open Data Processing Service, Open Data Processing services. This service is a PaaS (platform as a service) cloud computing platform of the product, ODPS ali has a large data storage and processing power to open up for external use, may be a large company, even entrepreneurial teams may be small and medium-sized enterprises (smes). At least pay hundreds of yuan can quickly complete online massive amounts of data processing.

ODPS project has started before 5 years, but about their future direction, platform positioning, and ali cloud on the relationship between other business has many not clear place, slow development. Until October of last year, ali cloud implementation of 5000 sets of single cluster project success and business across the room scheduling, on this basis, the development of ODPS began to accelerate, and formal business this year.

open before ODPS small business loan by ali, ali mother advertising platform, etc. Through ODPS on the seller’s credit evaluation, user click behavior prediction model training. Ali’s whole taobao and alipay, data warehouse, were set up on the ODPS, obviously this is the most important ali large data base software deployment. In order to verify the calculation of ODPS ability, ali invited genomics of ODPS gene sequencing, it takes less than one over ten of the traditional way; Invite the agency use of ODPS, regulatory drug flow all the way, solve the problem of counterfeit medicines. Now it will be verified ODPS officially opened out the commercial, is actually to ali to accommodate a double tenth and alipay platform grand open volume calculation and data capacity.

ali ambition: do big data arms dealer

in ali “data, platform and financial strategy, big data has become a ali’s top priority. Ma mentioned humans are into the era of DT in several occasions.

the most say American firms than for large data Google and Amazon, Google has the world-famous data center, cloud, based on the Colossus is quicker than the graphs of Caffeine, distributed storage Colossus is more advanced than GFS, and analysis of large data management tools Dremel, PowerDrill, Instant and Pregel, on the basis of these, Google can accurately forecast of the World Cup quarter-finals and flu trends prediction; Amazon besides AWS is the earliest IaaS platform, xuancheng last year by big data to predict the user’s purchasing behavior for shipment in advance.

and baidu, corresponding to China is ali, ali has cooperation with meteorological bureau to provide large data services, it is well known that climate science has always been a very typical huge amounts of data business, can also predict the user during double tenth a cash balance of consumption behavior, the novice network this open mode of the logistics system of intelligent dispatching also extremely dependent on big data. Baidu has the World Cup, tourism, forecast service of the university entrance exam, World Cup prediction accuracy than Google.

ali and baidu’s ideas are not the same, do is big data mart, ali has data can be submitted, ali, its capability of providing infrastructure and the third party may also help to analyze mining data owner. Ali was more in structures, data distribution, the collection and sharing of the underlying architecture. The underlying architecture is other companies use large data, ali will act as the role of the arms dealer. The ODPS is a part of belong to the underlying architecture, it can be for all companies and developers all large data capacity.

ODPS claims can be technology with top players such as Google and Amazon. In addition to the support of data size and the processing time is a bit less, ODPS also has more processing power, the future will be more powerful than Google BigQuery, support a rich SQL syntax (BigQuery maximum Like SQL statements), will also provide graphs programming ability to model and machine learning. Ali cloud last year achieved 5000 nodes ChanJiQun ability, and can realize the scheduling across the room and on the computing infrastructure to catch up with the level of foreign giants. ODPS is a big data technology embodied in the software level.

large data capacity highly concentrated into the model plant

after the industrial revolution, people of energy and materials for the full application, information revolution brings people than wealth essence is the largest specialized data, but in the past few decades of data utilization is still in the wild times. With big data concept of thorough popular feeling, enterprises, scientific research and education institutions, government departments of big data more seriously, mankind has entered the era of big data.

big data is landing, traffic, sports, health, education, electricity, and many other fields have been in the application of large data. Players in this field, however, are almost invariably technology giant, Google, Amazon, Microsoft, ali and companies such as baidu walked in the front. Operators, Banks and government departments have huge amounts of data but seldom does a typical application cases, some small and medium-sized companies either lack of data, or lack of ability of data mining.

open came up with the big data platform to help to solve the above problem. Steam engine in the era of energy supply is scattered, the invention of the alternating current (ac) power of the era of energy supply in the “cloud”, this progress stimulates the second industrial revolution for all to use electricity. Ali ODPS is actually the big data base ability from scattered to centralized, make big data more early into the era of industrial practice.

the layout of the ali recently reflected the open to the big data. The past ali cloud just provide the IaaS infrastructure and is now to IaaS + + SaaS, PaaS, such as small and medium-sized master data can be sent to ali cloud search index, so that small and medium-sized websites can easily have on-site search function. Ali cloud search train of thought is the prototype of the ODPS: are the data to the cloud.

FusionInsight huawei has launched a big data processing platform, it does not have any Internet accumulation, few consumer data, will tend to the underlying it is good at communication and equipment support. Baidu big data engine with three-piece: open cloud, the data factory and baidu brain, considerable strength in the field of general-purpose large data. Ali is good at is business data processing, infrastructure and software can be provided with capacity. The future of China’s big data capability, would like ali are highly concentrated, they are the “boss”, big data by other companies need to consume the large data capacity.

the author weibo @ Internet super, micro letter SuperSofter