Part 1: Critical Reply 170 words with references
According to Dr. Russom (2015) NoSQL DB, such as Hadoop, cannot replace RDB or Data Warehouse, at least not yet. It is because data scientists often view the NoSQL technology as an extension of the data warehouses with added support for unstructured and semi-structured data.
However, Russom (2015) indicated that data scientists often view NoSQL DB, or Hadoop, as a platform for advanced analytics addition to the already strong data warehouses with built-in support for reporting, OLAP and performance management.
Since data warehouses and traditional relational database does not readily support the third party or ad-hoc data without the lengthy process of structuring and defining the relationships, NoSQL DB comes in as a nifty extension to allow expedient analytics, improving the bottom lines, mining of social network data, and other miscellaneous data structures that traditional relational database does not support.
Additionally, Rossom mentioned (2015) that NoSQL may also be utilized as a modern dynamic tool for data archiving and record management, since columnar document database like MongoDB can store documents more effectively for quick querying and retrieval than regular windows folder storage locator with the Find option.
The features of NoSQL database allows dynamic characteristics for this archival purpose since the archival data/ documents can be prepared before loading and indexing, such as what MongoDB is best known for. However, Russom (2015) concluded that NoSQL databases are adding relational support and compatibility queries with SQL-structure syntax, so it is possible that one day NoSQL DB can actually replace RDB or data warehouse, to become a one solution fit all Database technology.
Part 2: Critical Reply 170 words with references
The first step toward successful Hadoop/EDW integration is to determine where Hadoop fits in the data warehouse architecture. As Hadoop is a family of products, each with multiple capabilities, there are multiple areas in data warehouse architectures where Hadoop products can contribute. Hadoop seems most compelling as a data platform for capturing and storing big data within an extended DW environment, in addition to processing that data for analytic purposes on other platforms.This approach allows firms to protect their investment in their respective EDW infrastructure and also extend it to accommodate the Big data environment.
The most prominent roles for Hadoop in EDW architectures are as follows.
Data staging. A considerable amount of data is processed in an EDW's staging area to prepare source data for specific uses (reporting, analytics) and for loading into specific databases (EDWs, data marts). Much of this processing is done by homegrown ETL tools. Hadoop thus allows organizations to deploy an extremely scalable and economical ETL environment. For example, one of the most popular ETL use cases is offloading heavy transformations, the "T" in ETL, from the data warehouse and into Hadoop.
The rationale behind this is because, for years, organizations have struggled to scale traditional ETL architectures. Specifically, many data integration platforms pushed the transformations down to the data warehouse, which is why today data integration in EDW architectures drives up to 80% of database capacity and resources, resulting in unsustainable spending, ongoing maintenance efforts, as well as poor user query performance. By shifting the "T" to Hadoop, organizations can dramatically reduce costs and free up database capacity and resources for faster user query performance.
Data archiving. Traditionally, enterprises had three options when it came to archiving data: leave it within a relational database, move it to an offline storage library, or purge it. Hadoop's scalability and low cost enable organizations to keep all data forever in a readily accessible online environment.
Schema flexibility. Relational DBMSs (used in Data Warehouse implementations) are well equipped in storing highly structured data, from ERP, CRM and other operational databases, to stable semi-structured data (XML, JSON). As a complement, Hadoop can quickly and easily ingest any data format, including evolving schema (as in A/B and multivariate tests on a website) and no schema (audio, video, images).
Processing flexibility. Hadoop's NoSQL approach is a more natural framework for manipulating non-traditional data types and enabling procedural processing, valuable in use cases such as time-series analysis and gap recognition. Hadoop also supports a variety of programming languages, thus providing more capabilities than SQL alone.
One way to augment and enhance the EDW in an organization with a Hadoop/big data cluster is as follows:
Continue to store summary structured data from OLTP and back office systems into the EDW.
Store unstructured data into Hadoop/NoSQL that does not fit nicely into tables. This means all the communication with customers from phone logs, customer feedbacks, GPS locations, photos, tweets, emails, text messages, etc. can be stored in Hadoop.
Co-relate data in EDW with the data in the Hadoop cluster to get better insight about customers, products, equipment, etc. Organizations can now run ad-hoc analytics and clustering and targeting models against this co-related data in Hadoop, which is otherwise computationally very intensive.
Part 3: Critical Reply 150 words with references
I looked at the company Foghorn, which develops solutions that can process and analyze data on the "edge" of the network - i.e. where the data is actually being collected. They have a number of job openings for engineers. These jobs all require experience in the C++ language. In addition, the applications that these employees are expected to develop have different purposes. For example, one of them seems to be very metrics-based, with development of an application that measures latency and other performance measurements. I would assume that this applicant would need to be very experienced in networking technologies. Other engineers are tasked with designing the more customer-facing applications which display the analytics being performed on the collected data.
That brings me to other technical-oriented openings in the company - UI designers. These positions call for applicants that have experience in JavaScript, HTML5, and CSS. They will also be interpreting data that is in JSON format, which is consistent with the data storage in many NoSQL databases we have been studying for the past two weeks. On the other hand, they also call for experience in some languages/protocols that I've never heard of, such as AngularJS, SASS, AJAX, and REST.
Part 4: Critical Reply 150 words with references
The Internet of Things technologies in the Postscapes examples do have elements in common. Most of them piggy-back on existing connectivity solutions, such as Wi-Fi or Bluetooth. Wearable technology often utilizes ones smart phone in order to transmit or analyze data, as smart phones already have the built in hardware to for the IoT device to take advantage of. IoT devices can be very low profile when connected to a more powerful piece of hardware such as a smart phone, to which it can offload data storage, transmission to external servers, and computational-intensive processes. More stationary devices in the home or business can utilize the Wi-Fi connection that most of us already have. Sensors that are located in more remote locations can use 3G or 4G mobile networks that telecom companies have already installed.
The ability of IoT devices to easily (and transparently) integrate with existing communication technologies is key to its adoption. As seen in the articles and advertisements where the phone interface is seen right next to the device itself, smart phone integration is seen as a benefit for users who are able to see graphical analysis of their data, while the underlying technology that allows for that is something that users hardly even have to consider.
For the job search, I looked at local (Washington, D.C. area) jobs at SAP. SAP offers software platforms that allow for management of IoT data. Predictably, there were a lot of jobs in the sales area. As for more technical-oriented jobs, they have openings for software design engineers. These openings call for experience in JavaScript, CSS, HTML5, and SQL. In addition to the design engineers, there are support analyst positions. These positions call for experience in troubleshooting software issues, and detail the myriad ways that they require their analysts to retrieve and interpret logs.