ES SOFTWARE

APACHE SPARK popular open source platform -TECHNOLOGY-RESEARCH PAPER-SOFTWARE SALES SERVICE


open-source cluster-computing framework. Apache Spark, the unified analytics engine, has seen rapid adoption by enterprises across a wide range of industries. Internet powerhouses such as Netflix, Yahoo, and eBay have deployed Spark at massive scale, collectively processing multiple petabytes of data on clusters of over 8,000 nodes.

Mllib: Machine learning in apache spark
free download

Abstract Apache Spark is a popular open-source platform for large-scale data processing that is well-suited for iterative machine learning tasks. In this paper we present MLlib, Sparks open-source distributed machine learning library. MLlib provides efficient

Beyond Hadoop MapReduce Apache Tez and Apache Spark
free download

ABSTRACT Hadoop MapReduce has become the de facto standard for processing voluminous data on large cluster of machines, however this requires any problem to be formulated into strict three-stage process composed of Map, Shuffle/Sort and Reduce. Lack

linalg: Matrix computations in apache spark
free download

Abstract We describe matrix computations available in the cluster programming framework, Apache Spark . Out of the box, Spark comes with the mllib. linalg library, which provides abstractions and implementations for distributed matrices. Using these abstractions, we

Static and dynamic big data partitioning on apache spark .
free download

Abstract. Many of todays large datasets are organized as a graph. Due to their size it is often infeasible to process these graphs using a single machine. Therefore, many software frameworks and tools have been proposed to process graph on top of distributed

Alternating direction method of multipliers implementation using Apache Spark
free download

Many application areas in optimization have benefited from recent trends towards massive datasets. Financial optimization problems ingest decades of fine-grained stock history and recent energy grid optimization techniques optimize hundreds of millions of variables

Real-time News Recommendations using Apache Spark .
free download

Abstract. Recommending news articles is a challenging task due to the continuous changes in the set of available news articles and the context dependent preferences of users. Traditional recommender approaches are optimized for analyzing static data sets. In news

Scalable sde filtering and inference with apache spark
free download

Abstract In this paper, we consider the problem of Bayesian filtering and inference for time series data modeled as noisy, discrete-time observations of a stochastic differential equation (SDE) with undetermined parameters. We develop a Metropolis algorithm to sample from the

Efficient big data analysis with Apache spark in HDFS
free download

Abstract With the size of data increasing each day, the traditional methods of data processing have become inefficient and time consuming. Today, Facebook, Google, Twitter are generating Petabytes of data each day. This large amount of data is given the term Big

When apache spark meets FPGAs: a case study for next-generation DNA sequencing acceleration
free download

FPGA-enabled datacenters have shown great potential for providing performance and energy efficiency improvement. In this paper we aim to answer one key question: how can we efficiently integrate FPGAs into stateof-the-art big-data computing frameworks like

A review study of Apache Spark in Big data processing
free download

ABSTRACT Why Spark becomes a hot topic in Big Data analytics Is really Apache Spark going to replace Hadoop If we involved seriously into Big Data analytics, then, should we really care about Spark Apache Spark is a lightning-fast cluster computing designed for fast

Performance evaluation of apache spark on cray xc systems
free download

Abstract We report our experiences in porting and tuning the Apache Spark data analytics framework on the Cray XC30 (Edison) and XC40 (Cori) systems, installed at NERSC. Spark has been designed for cloud environments where local disk I/O is cheap and performance is

A scalable, secure and real-time healthcare analytics framework with apache spark
free download

Abstract A Big Data analytics framework with related computing technologies can process huge amounts of real-time data to obtain tremendous insights for effective clinical decision making in the healthcare research. In this paper, we propose a healthcare analytics

Performance improvement in apache spark through shuffling
free download

Abstract Apache Spark is a fast and general engine for large-scale data processing. Shuffle Phase refers to the partitioning and aggregation of data during an all-to all operations. Spark shuffle performance is improved in Sort-based Shuffle. Spark Shuffle

Modeling and Simulating Apache Spark Streaming Applications
free download

Abstract Stream processing systems are used to analyze big data streams with low latency. The performance in terms of response time and throughput is crucial to ensure all arriving data are processed in time. This depends on various factors such as the complexity of used

Exploring scalable implementations of triangle enumeration in graphs of diverse densities: Apache spark vs. gpu
free download

Graphs are powerful tools for modeling. Model social interaction: Friendship graphs Social networks Collaboration/Co-authorship graphs Phone call graphs Model computer networks: WWW (pages linking to other pages) WWW (hardware linking to other hardware) Model data moving through

An information theory-based feature selection framework for big data under apache spark
free download

Abstract With the advent of extremely high-dimensional datasets, dimensionality reduction techniques are becoming mandatory. Of the many techniques available, feature selection is of growing interest for its ability to identify both relevant features and frequently repeated

Apache Spark : Fast and Easy Data Processing
free download

Apache Spark : Fast and Easy Data Processing SNIA Analytics and Big Data

Big Data Analysis: Comparision of Hadoop MapReduce and Apache Spark
free download

Abstract In recent years, the rapid development of the Internet, Internet of Things, and Cloud Computing have led to the explosive growth of data in almost every industry and business area. Big data has rapidly developed into a hot topic that attracts extensive attention from

On realizing rough set algorithms with apache spark
free download

Page 1. On Realizing Rough Set Algorithms with Apache Spark Innovation Center for Big Data and Digital Convergence Department of University 135 Yuan-Tung Road, Chung-Li, TAIWAN 32003

Performance Comparison of Map Reduce and Apache Spark on Hadoop for Big Data Analysis
free download

Abstract With the unremitting advancement of internet and IT, tremendous growth of data has been observed. Data creation occurring at very fast pace, referred as big data, is a trending term these days. Big Data has been the topic of fascination for Computer Science

Scalability Potential of BWA DNA Mapping Algorithm on Apache Spark .
free download

Abstract This paper analyzes the scalability potential of embarrassingly parallel genomics applications using the Apache Spark big data framework and compares their performance with native implementations as well as with Apache Hadoop scalability. The paper uses the

Novel Apache Spark based Algorithm to Solve Dirichlet Problem for Poisson Equation in 3D Computational Domain
free download

Abstract: Parallel computations are essential tool in solving large-scale computationally demanding problems. Due to large diversity and heterogeneity of the currently available parallel processing techniques and paradigms it is usually difficult to find the right solution

Decision Tree Learning and Regression Models to Predict Endocrine Disruptor Chemicals-A Big Data Analytics Approach with Hadoop and Apache Spark
free download

Abstract-Predictive toxicology calls for innovative and flexible approaches to mine and analyse the mounting quantity and complexity of data used in it. Classification and regression based machine learning algorithms are used in this study in order to

Implementing a GPU-based Machine Learning Library on Apache Spark
free download

As data storage becomes increasingly commoditized, companies are collecting transactional records on the order of several petabytes that are beyond the ability of typical database software tools to store and analyze. Analysis of this big data can yield business

Closest-pairs query processing in apache spark
free download

Abstract Processing of spatial queries when the datasets involved are big can be accomplished efficiently in a parallel and distributed environment. The (K) Closest-Pair (s) Query, KCPQ, is a common query in many real-life applications involving geographical, or,

Benchmarking Apache Spark with Machine Learning Applications
free download

Abstract We benchmarked Apache Spark with a popular parallel machine learning training application, Distributed Stochastic Gradient Descent for Matrix Factorization and compared the Spark implementation with alternative approaches for communicating model

Spatio-Temporal Hotspot Computation on Apache Spark (GIS Cup)
free download

ABSTRACT Large quantities of mobility data are produced by people and vehicles daily. Mining and analysis of patterns, such as hotspots, in this data can serve to improve location- based services. However, due to the massive amount of information, efficient techniques are The Spread of cameras and sensors and cloud technologies enable us to obtain life logs at ordinary homes and transmit the captured data to a cloud for life log analysis. However, the amount of processing for video data analysis in a cloud drastically increases when a very

Accelerating Apache Spark with Fixed Function Hardware Accelerators Near DRAM and NVRAM
free download

Our recent work on performance characterization of Apache Spark on a Scale-up Server shows that performance of Spark workloads is limited by the latency of frequent accesses to DRAM and as the input data size is enlarged, the DRAM capacity becomes the bottleneck

MPIgnite: An MPI-Like Language for Apache Spark
free download

ABSTRACT Scale-out parallel processing based on MPI is a 25-year-old standard with at least another decade of preceding history of enabling technologies in the High Performance Computing community. Newer frameworks such as MapReduce, Hadoop, and Spark

Performance Evaluation of Large Table Association Problem Implemented in Apache Spark on Cluster with Angara Interconnect
free download

Abstract. In this paper we consider an association problem with constraints for two dynamically enlarging tables. We consider a base full association algorithm and propose a partial association algorithm that improves efficiency of the base algorithm. We implement [DOC]

MOBILE BIG DATA ANALYTICS USING DEEP LEARNING AND APACHE SPARK WITH miVLAD
free download

Abstract The proliferation of mobile devices, such as smartphones and Internet of Things gadgets, has resulted in the recent mobile big data era. Collecting mobile big data is unprofitable unless suitable analytics and learning methods are utilized to extract

A Review Document on Apache Spark for Big Data Analytics with Case Studies
free download

ABSTRACT Evolution in technology has given rise to usage of new methods for collecting data. On the other side the size of the data that is getting collected is of huge size which is categorized as Big Data. Big Data has three main characteristics namely Volume, Velocity

Distributed method for crossmatching of astronomical catalogs based on apache spark platform
free download

Copyright 2016 for the individual papers by the papers authors. In the article horizontally- scalable algorithm for matching astronomical catalogs, based on Apache Spark distributed computing platform, is proposed. The method provides the necessary accuracy and good

Good parallel software development practices. Apache Spark case
free download

Abstract Recently, Spark as data processing engine, gained huge popularity because of better performance in terms of the speed. Developers of Spark claim that it may outperform Hadoop MapReduce in 100 times in memory and 10 times on disk . This paper outlines

Apache Spark and Big Data Analytics for Solving Real World Problems
free download

ABSTRACT Big Data analysis is having an impact on every industry today. Industry leaders are capitalizing on these new business insights to drive competitive advantage. Apache Hadoop is the most common Big Data Framework, but the technology in evolving rapidly

Sampling Selection Strategy for Large Scale Deduplication in a Distributed System Using Apache Spark
free download

Abstract The generation of information from a wide range of sources has opened opportunities for the emergence of several new applications such as digital libraries, media streaming etc. that presuppose high quality data to provide reliable services. Data quality is

Econometric modeling of panel data using parallel computing with Apache Spark
free download

Summary The aim of this article is to provide a method for determining the fixed effects estimators using MapReduce programming model implemented in Apache Spark . From many known algorithms two common approaches were exploited: the within transformation

Performance Comparison of Apache Spark and Tez for Entity Resolution
free download

Abstract Entity Resolution is among the hottest topics in the field of Big data. It finds duplicates in datasets, which actually belong to same entity in the real world. Algorithms that perform Entity Resolution are computation intensive and consume a lot of time especially for

RefineOnSpark: a simple and scalable ETL based on Apache Spark and OpenRefine
free download

Abstract Over the last decade, big data became a catch-all term for anything that handles non-trivial sizes of data. It is used to describe the industry challenge posed by having data harvesting abilities that far outstrip the ability to process, interpret and act on that data.

SparkBurst: An Efficient and Faster Sequence Mapping Tool on Apache Spark Platform
free download

Abstract Next generation sequencing (NGS) technologies are generating a huge amount of genetic data due to which conventional single-processor sequence alignment tools are unable to keep trace with them. Therefore, cloud computing and MapReduce frameworks,

Parallel Maritime Traffic Clustering Based on Apache Spark
free download

Abstract Maritime traffic patterns extraction is an essential part for maritime security and surveillance and DBSCANSD is a density based clustering algorithm extracting the arbitrary shapes of the normal lanes from AIS data. This paper presents a parallel DBSCANSD

Apache Spark Streaming
free download

Abstract This paper is the result of theAdvanced Database Systems seminar at the University of Applied Sciences in Rapperwil. The key point is to explain and understand the function of data stream management systems. The paper is split into two parts to cover a

Data Model Optimization for Reducing Computational Cost at Apache Spark
free download

ABSTRACT As the performance of distributed parallel processing on big data is considered as a main concern, Apache Spark the most prevalent open source based distributed processing engine has triggering much interest for performance optimization. According to

Transparent Avoidance of Redundant Data Transfer on GPU-enabled Apache Spark
free download

Abstract This paper presents an extension to IBMSparkGPU, which is an Apache Spark framework capable of compute-or memoryintensive tasks on a graphics processing unit (GPU). The key contribution of this extension is an automated runtime that implicitly avoids

Hot Spot Detection to perform Geo Spatial Temporal Data Operations using Apache Spark
free download

ABSTRACT This paper briefly describes the various phases of the project implemented using GeoSpark, a cluster computing framework for processing large scale spatial data. The project is divided into three phases which included working with different technologies and

Correlating Apache Spark and Map Reduce with Performance Analysis using K-Means
free download

ABSTRACT Big Data has for some time been the point of interest for Computer Science devotees around the globe, and has increased much more conspicuousness in the late times with the nonstop blast of information coming about because of any semblance of

Attribute Reduction: An Implementation of Heuristic Algorithm using Apache Spark
free download

Abstract-Most weather event occurs in the troposphere, the lowest level of atmosphere. Weather describe the degree of the condition as hot or cold, minimum and maximum temperature, clear or cloudy, atmospheric pressure. This degree changes at instant time due

Sideloading-Ingestion Of large point clouds into the apache spark big data engine
free download

In the geospatial domain we have now reached the point where data volumes we handle have clearly grown beyond the capacity of most desktop computers. This is particularly true in the area of point cloud processing. It is therefore naturally lucrative to explore established

MAKING ONLINE RECOMMENDATIONS MADE EASY BY APACHE SPARK
free download

ABSTRACT With the advent of social networks, forums, and blogs, the amount of data on the Web has increased rapidly, resulting in an information explosion. Using the Internet, users make purchases, listen to music, or watch a movie; then later on, they make comments

Implementing K-Means for Achievement Study between Apache Spark and Map Reduce
free download

ABSTRACT Huge Data has for quite some time been the subject of enthusiasm for Computer Science fans around the globe, and has increased much more conspicuousness in the later times with the constant blast of information coming about because of any

Parallelizing an Experiment to Decide Shellability on Bipartite Graphs Using Apache Spark
free download

Abstract Graph shellability is an NP problem whose classification either in P or in NP- complete remains unknown. In order to understand the computational behavior of graph shellability on bipartite graphs, as a particular case, it could be useful to develop an efficient

Is Distributedworth it Benchmarking Apache Spark with Mesos
free download

Abstract A lot of research focus lately has been on building bigger distributed systems to handle Big Dataproblems. This paper examines whether typical problems for web-scale companies really benefits from the parallelism offered by these systems, or can be handled

An Investigation on Extensive Graphs of Distributed Prims Minimum Spanning Tree Construction Using Apache Spark
free download

Abstract: Minimum spanning trees are a standout amongst the most essential primitives utilized as a part of graph algorithms. They discover applications in various fields going from scientific categorization of Network design, Approximation algorithms for NP-hard problems,

Pipelined execution of stages in Apache Spark
free download

Abstract This dissertation investigates the efficiency of a fundamental, low-level building block of modern big-data processing platforms, like Apache Spark . This type of platforms supports complex data analysis tasks by allowing data scientists to express arbitrary data

Big Data and Apache Spark : A Review
free download

Abstract Big Data is currently a very burning topic in the fields of Computer Science and Business Intelligence, and with such a scenario at our doorstep, a humungous amount of information waits to be documented properly with emphasis on the market. By market, we

Towards Distributed Model Analytics with Apache Spark
free download

Abstract: The growing number of models and other related artefacts in model-driven engineering has recently led to the emergence of approaches and tools for analyzing and managing them on a large scale. The framework SAMOS applies techniques inspired by

Apache Spark on HPC clusters
free download

Data Analytics Tools at OSC Case study Empowering Clients:

A review of the Hadoop ecosystem exploring the TFOCS optimization solver utilizing the data processing engine of Apache Spark
free download

Abstract The volume of data generated by different types of sources such as social media networks, financial transactions, books, video even any kind of sensors are increasing exponentially in terms of their volume, variety and velocity. In general, various mathematical

IMPLEMENTATION OF POD AND DMD METHODS IN APACHE SPARK FRAMEWORK FOR SIMULATION OF UNSTEADY TURBULENT FLOW IN THE MODEL
free download

Abstract. The paper is devoted to modelling and analysis of unsteady turbulent flow in a model combustor (channel) using LES (Large Eddy Simulation). Simulations were provided for 2D and 3D cases on different grids of a flow in a channel with rearward facing step. The
BIG DATA
CLOUD COMPUTING
IOT-INTERNET OF THINGS
ROBOTICS
CONTACT US

NEW IEEE PROJECTS

IEEE PROJECT PAPERS FOR CSE
IEEE PROJECTS DOWNLOAD
IEEE PROJECTS FOR EEE
IEEE PROJECT PAPERS FOR ECE
IEEE PROJECTS FOR ECE 2016
IEEE PROJECTS FOR ECE 2017
IEEE PROJECTS 2017
IEEE MINI PROJECTS