Question

For this assignment, you will formulate a plan to deal with large data sets. This assignment is based off a fictitious scenario where you as the analyst must complete an ETL process on a large dataset. You will need to consider the issues faced when dealing with Big Data and draft a plan to deal with it. The submitted paper should be a minimum of one page and include the following detail:

Address why Big Data creates challenges for ETLs.

List and describe some of the pain points for dealing with large data sets.

Prescribe a plan (remedy) for dealing with these pain points.

Answer 1

Answer)

Big Data creates challenges for ETLs:

1) Data loss during ETL testing

2) Duplicate data as well as incompatibility

3) Data volumne and complexity

With respect to large data sets the pain points are as follows:

1) Creates traffic and leads to connection glitches

2) Slows down the data base

3) Krell Problem

4) Physically impossible data

5) text field formatting

To deals with these pain points one can do the following :

1) Change the column / file name

2) Add data codes

3) Text field formatting

4) Data analysis.

Hit like/ upvote if you find the answer useful. Your response is important to us and is much needed.

Hope this answer helps. Happy to help. :)