Big Data Tools & Architecture Paper

Overview

This week is for installation of R and R Studio. This also marks the beginning of our exploration of what ‘Big’ means in this context.

Weekly Objectives:

Successfully install R

Successfully install R Studio

Describe R Studio windows and tabs

Define the ‘Big’ in Big Data

Install sparlyr

Install the Spark Cluster on a local machine

Connect to the Spark Cluster

Required Readings

Required Readings

The text is Mastering Spark with R. It is an O’Reilly book. Read chapter 1 and 2 the Introduction

Week One Assignment: Install R and R Studio

Install R and R Studio.

Submit a Word or .pdf document with screen shots of R Studio where you have created a vector of three words. Whenever you are asked to submit a screenshot, include either a sliver of your desktop or a timestamp from your desktop. Always repeat the question you are answering.

Please note that all code assignments must be submitted as a screenshot with a slice of your desktop showing the timestamp.

If the time and date are not visible, you will be graded 0.

Put the screenshots in a word document, make sure to comment the code (explain what it does)  and interpret the graph if applicable(explain what its depicting)

All assignments will go through SafeAssign. Your score should be less than 30 and you will only be allowed 2 attempts.

Please email your instructor if you have any questions.

Due by Sunday 11:59 PM EST.

Week One Assignment: Spark Install

Following the instrutions in the book, install the sparklyr library and a Spark cluster on your local machine.  The instructions in the book contemplate that you will be using Windows. If you are not running Windows, use a virtual machine. If you are using your employer’s equipment, you may enounter a trusted domain error. This will also require that you install a virtual machine. The code in the book assumes Spark version 2.3 and Java 8. You are free to use any version, of course, but the code may not run and you may have to modify it to complete the assignment. Be sure to set your system to be using Java 8. You can use the Sys.setenv() function to set JAVA_HOME with the path to jre1.8.0_291 (the latest release of Java 8)

Submit a Word document with screenshots from your computer showing R studio and a time stamp. Your screenshots should show the console in R studio with the following:

1. Spark available versions

2. Spark installed versions

3. a successful connection to the local Spark master.

Week One Discussion

Week One Discussion

Write at least 500 words on what ‘Big’ means in Big Data. What exposure have you had to Big Data?

Use at least three sources. Include at least 3 quotes from your sources enclosed in quotation marks and cited in-line by reference to your reference list.  Example: “words you copied” (citation) These quotes should be one full sentence not altered or paraphrased. Cite your sources using APA format. Use the quotes in your paragraphs.

Write in essay format not in bulleted, numbered or other list format.

Reply to two classmates’ posting in a paragraph of at least five sentences by asking questions, reflecting on your own experience, challenging assumptions, pointing out something new you learned, offering suggestions. Make your initial post by   Thursday 11:59 PM EST. Respond to two of your classmates by   Sunday 11:59 PM EST.     These peer responses are not ‘attaboys’. If you are going to say something like ‘great post’, you MUST explain what makes it great. This is a powerful way to let the writer know that you’ve read and thought about his/her post. Be sure to use the name of the author of the post to whom you are responding.

Cite your sources in a clickable reference list at the end. Do not copy without providing proper attribution (quotation marks and in-line citations). Write in essay format not in bulleted, numbered or other list format.

It is important that you use your own words, that you cite your sources, that you comply with the instructions regarding length of your post and that you reply to two classmates in a substantive way (not ‘nice post’ or the like).  Your goal is to help your colleagues write better. Do not use spinbot or other word replacement software. Proof read your work or have it edited. Find something interesting and/or relevant to your work to write about.  Please do not submit attachments unless requested