BBVA API Market
At the BBVA Innovation Center on Plaza Santa Bárbara in Madrid, Jorge López-Malla, Big Data Architect at Stratio, explained why so many great things have been heard for some time about Spark, which he had no access to until he started working for his present company. Stratio is looking for developers, and this is what they said during the presentation by their Human Resources representatives.
The event, which promised to give answers to key issues for development in the Apache framework -how to combine SparkSQL processes with others launched from the Spark Core or the algorithm application of MILib to real-time logic- began like a recent history lesson.
Since the concept of Big Data was born in 2003, in a paper by Google on distributed file processing, we had to wait until 2006 for the Yahoo! team to launch Hadoop, which ended up constituting the basis on which operations with Big Data would take shape.
The problem, according to López-Malla, is that Hadoop emerged in response to a type of problem different from that faced today by a developer who works with distributed file processing. Technology has changed in 10 years, but so has the market and the demand for software.
Flink (also open source) and Spark emerged in response to today’s problems and, according to López-Malla, the latter “in not the future, but the present of Big Data”. The ground-breaking feature of Spark is its processing speed. It is therefore “an evolution of Hadoop and its paradigm”, but with the advantage of offering a performance 10 to 100 times greater than any distributed computing platform.
Everything is based on RDDs, or “collections of distributed collections”, focused on processing in partitions. These partitions, which are independent from each other, enable the workflow to continue with no interruptions without taking into account what happens in the others.
If Hadoop’s core improved, the modules did not benefit from that improvement. Spark changes this radically. Programmers now benefit from the fact that Spark has a single API for everything.
Three of Spark’s most popular modules were present during the afternoon at the BBVA Innovation Center. Spark SQL (for querying structured data with SQL language or an API), Spark Streaming (for managing data in real time instead of by batches) and MILib, for providing Spark with functionalities related to machine learning.
The full presentation by Jorge López-Malla, with visual and operation examples, is available on the BBVA Innovation Center’s YouTube channel, where the video that you can see embedded below can be found along with many others.
Follow us on @BBVAAPIMarket
In the dynamic world of payments, a new star has emerged in recent years: Buy Now Pay Later (BNPL), i.e. short-term financing that allows you to buy now and pay later. This model allows businesses to purchase goods or services and pay for them in installments, often interest-free, making it an attractive alternative to credit […]
BBVA and Vecttor, Cabify’s subsidiary engaged in managing vehicles with drivers, have entered into an alliance that saves time and provides security to the company and its drivers. The collaboration allows drivers to deposit cash collections at any BBVA ATM and Vecttor to automatically reconcile this activity from their accounts with those in the company’s […]
BBVA has been recognized by Global Finance as the bank with the best global open banking offer for companies. This award comes on top of 12 other recognitions the magazine has bestowed on the company, such as the best bank for corporate clients and the one recognizing its AI factory as one of the best […]
