// Big Data Analytics

Big Data Analytics

We help our clients to unlock the value of their data by providing business analytics dashboards, digital solutions and services to address specific business challenges through digital capabilities as a competitive advantage.

“Big data is a term for data sets that are so large or complex that traditional data processing application softwares are inadequate to deal with them. Challenges include capture, storage, analysis, data curation, search, sharing, transfer, visualization, querying, updating and information privacy. The term “big data” often refers simply to the use of predictive analytics, user behavior analytics, or certain other advanced data analytics methods that extract value from data, and seldom to a particular size of data set. “There is little doubt that the quantities of data now available are indeed large, but that’s not the most relevant characteristic of this new data ecosystem.” Analysis of data sets can find new correlations to “spot business trends, prevent diseases, combat crime and so on.” Scientists, business executives, practitioners of medicine, advertising and governments alike regularly meet difficulties with large data-sets in areas including Internet search, finance, urban informatics, and business informatics. Scientists encounter limitations in e-Science work, including meteorology, genomics, connectomics, complex physics simulations, biology and environmental research.”
Source en.wikipedia.org/wiki/Big_data

Data Processing / Data Flow:

  • The data are sent from the data tracking application to the chosen data storage solution, data processing system, data warehouse, etc. through the integration of the chosen API
  • The data processing software differs according to the analytic task to be carried out
    • historic analysis where a delay of hours or days is no problem
      • Apache Hadoop
    • real time data processing of events where context and sums are not important
      • Apache Storm
    • mixed processing, where real time processing is not important and slight delays are OK
      • Apache Flink
        • historic processing possible, with slight delays
        • “fast” processing of events possible if deep analysis is required, but not in “real time” with delays of a couple of seconds
        • can run hadoop and storm scripts, so it’s good for later upgrades and expansions, when you have started a specialised solution
  • for Visualising the results the data has to be transferred to other systems
    • Elasticsearch+Kibana
      • specialised on log-analyses (processing of text-based events)
    • InfluxDB+Grafana
      • Grafana can also read from other data storages (e.g. Elasticsearch)
      • the visualisation of Grafana is based on the display and sum up of figures respectively the counting of events.
      • text based search & analytics can be conducted
      • text based data must be converted to meaningful numbers for data processing
    • Apache Cassandra+Apache Zeppelin
      • Apache Zeppelin Data can be read directly from Hadoop’s HDFS or Flink in which case cassandra would not be necessary

Application Scenarios

  • Big Data Analytics
  • Business Analytics
  • Learning Analytics
  • Game Analytics

Technologies

We provide expertise in platforms including:

and other big data tools and technologies:

RAGE Analytics

“RAGE, Realising an Applied Gaming Eco-system, aims to develop, transform and enrich advanced technologies from the leisure games industry into self-contained gaming assets that support game studios at developing applied games easier, faster and more cost-effectively. These assets will be available along with a large volume of high-quality knowledge resources through a self-sustainable Ecosystem, which is a social space that connects research, gaming industries, intermediaries, education providers, policy makers and end-users.”
Source rageproject.eu

  • Client-side interaction tracking asset: This asset provides a ready-to-use in-game component to track the player’s interactions with the game, so that they can be submitted to a server for further analysis. Available at github for unity and JS; client-side part of rage-analytics.
  • Server-side interaction storage and analytics asset: This asset offers a ready-to-deploy server-side implementation of a data collection and storage service. It takes care of authentication and supports current standards for exchange of interaction data. Available at github; part of rage-analytics
  • Server-side Dashboard and Analysis Asset: This asset provides a ready-to-use user-friendly website where stakeholders (from instructors to policy-makers) can configure a dashboard with different visualizations of the data gathered by the different data collection assets provided by RAGE. In particular, this asset is used to display customized analytics data to game developers, teachers, students, and educational managers. Available at github; part of rage-analytics
  • Server-side Authorization and Authentication Asset: This asset provides a central location where clients can authenticate and locate server-side assets, including analytics. Servers can also register to locate other servers and be locatable by clients, restricting access using configurable roles. This asset is a spin-off from the Server-Side Dashboard and Analytics asset, since multiple server-side assets want single sign-on while having nothing to do with analytics.

Available at github; part of rage-analytics