• Description

This course constitutes an introduction to Big Data systems and technologies. The first block addresses the technological structure behind Big Data projects which includes relevant aspects such as the distributed calculation and storage system or the management of the cluster's hardware resources. The next block addresses the two main models of distributed processing: batch and stream processing for simple and complex events. We will see the main functions and characteristics of the most widely used frameworks today, paying special attention to the two great standards: Apache Hadoop and Apache Spark. Finally, in the last block of the subject, we will review the main data analysis libraries, including machine learning, graph analysis and massive data visualization paying special attention when this methods are applied to Big Data problems.

• The subject within the syllabus as a whole

This subject corresponds to an optional course within the University master's degree in Data Science and in Computational Engineering and Mathematics (joint URV, UOC).

• Professional fields to which it applies

The course provides knowledge useful in different professional fields related to the development of software, data science, and machine learning on systems that require the use of big data technology. The course will also be useful for the management or consulting of projects based on Big Data systems, among others.

• Prior knowledge

This course requires students to have basic to intermediate programming skills in Python since 90% of the course will be based on that language. The remaining 10% will be done in Java. Basic to intermediate knowledge about data analysis, machine learning, and computer networking are also assumed.

The course also includes case studies and autonomous information research, it is advisable for the student to be familiar with the search information sources over the internet, the analysis of quantitative and qualitative information, the ability to synthesize and obtain conclusions, as well as possess certain written communication skills.

• Information prior to enrolment

None.

• Learning objectives and results

The objectives that student will achieve through this course are the following:

• Understand the concepts and formal definitions associated with Big Data and related concepts.
• Identify the necessary technological elements in any project related to Big Data.
• Be able to decide about the most appropriate methodologies for the implementation of Big Data systems.
• Learn about the main tools available in the Big Data ecosystem, especially the Apache Hadoop and Apache Spark ecosystems.
• Build machine learning models to generate knowledge as a result Big Data analysis.
• Know the basic operation of the main Big Data tools and frameworks, such as HDFS or Apache Spark.

• Content

• View the UOC learning resources used in the subject

• Guidelines on assessment at the UOC

Assessment at the UOC is, in general, online, structured around the continuous assessment activities, the final assessment tests and exams, and the programme's final project.

Assessment activities and tests can be written texts and/or video recordings, use random questions, and synchronous or asynchronous oral tests, etc., as decided by each teaching team. The final project marks the end of the learning process and consists of an original and tutored piece of work to demonstrate that students have acquired the competencies worked on during the programme.

To verify students' identity and authorship in the assessment tests, the UOC reserves the right to use identity recognition and plagiarism detection systems. For these purposes, the UOC may make video recordings or use supervision methods or techniques while students carry out any of their academic activities.

The UOC may also require students to use electronic devices (microphones, webcams or other tools) or specific software during assessments. It is the student's responsibility to ensure that these devices work properly.

The assessment process is based on students' individual efforts, and the assumption that the student is the author of the work submitted for academic activities and that this work is original. The UOC's website on academic integrity and plagiarism has more information on this.

Submitting work that is not one's own or not original for assessment tests; copying or plagiarism; impersonation; accepting or obtaining any assignments, whether for compensation or otherwise; collaboration, cover-up or encouragement to copy; and using materials, software or devices not authorized in the course plan or instructions for the activity, including artificial intelligence and machine translation, among others, are examples of misconduct in assessments that may have serious academic and disciplinary consequences.

If students are found to be engaging in any such misconduct, they may receive a Fail (D/0) for the graded activities in the course plan (including final tests) or for the final grade for the course. This could be because they have used unauthorized materials, software or devices (such as artificial intelligence when it is not permitted, social media or internet search engines) during the tests; copied fragments of text from an external source (the internet, notes, books, articles, other students' work or tests, etc.) without the corresponding citation; purchased or sold assignments, or undertaken any other form of misconduct.

Likewise and in accordance with the UOC's academic regulations, misconduct during assessment may also be grounds for disciplinary proceedings and, where appropriate, the corresponding disciplinary measures, as established in the regulations governing the UOC community (Normativa de convivència).

In its assessment process, the UOC reserves the right to:

• Ask students to provide proof of their identity as established in the UOC's academic regulations.
• Ask students to prove the authorship of their work throughout the assessment process, in both continuous and final assessments, through a synchronous oral interview, of which a video recording or any other type of recording established by the UOC may be made. These methods seek to ensure verification of the student's identity, and their knowledge and competencies. If it is not possible to ensure the student's authorship, they may receive a D grade in the case of continuous assessment or a Fail grade in the case of the final assessment.

Artificial intelligence in assessments

The UOC understands the value and potential of artificial intelligence (AI) in education, but it also understands the risks involved if it is not used ethically, critically and responsibly. So, in each assessment activity, students will be told which AI tools and resources can be used and under what conditions. In turn, students must agree to follow the guidelines set by the UOC when it comes to completing the assessment activities and citing the tools used. Specifically, they must identify any texts or images generated by AI systems and they must not present them as their own work.

In terms of using AI, or not, to complete an activity, the instructions for assessment activities indicate the restrictions on the use of these tools. Bear in mind that using them inappropriately, such as using them in activities where they are not allowed or not citing them in activities where they are, may be considered misconduct. If in doubt, we recommend getting in touch with the course instructor and asking them before you submit your work.

• View the assessment model