DATA2001

Big Data and Data Diversity

This course focuses on methods and techniques to efficiently explore and analyse large data collections. Where are hot spots of pedestrian accidents across a city? What are the most popular travel locations according to user postings on a travel website? The ability to combine and analyse data from various sources and from databases is essential for informed decision making in both research and industry. Students will learn how to ingest, combine and summarise data from a variety of data models which are typically encountered in data science projects, such as relational, semi-structured, time series, geospatial, image, text. As well as reinforcing their programming skills through experience with relevant Python libraries, this course will also introduce students to the concept of declarative data processing with SQL, and to analyse data in relational databases. Students will be given data sets from, eg. , social media, transport, health and social sciences, and be taught basic explorative data analysis and mining techniques in the context of small use cases. The course will further give students an understanding of the challenges involved with analysing large data volumes, such as the idea to partition and distribute data and computation among multiple computers for processing of 'Big Data'. 本课程重点介绍有效探索和分析大数据集的方法和技巧。整个城市的行人事故热点在哪里？根据旅游网站上的用户帖子，最受欢迎的旅游地点是什么？结合和分析来自各种来源和数据库的数据的能力对于研究和工业中的知情决策至关重要。 学生将学习如何摄取，组合和汇总来自各种数据模型的数据，这些数据模型通常在数据科学项目中遇到，例如关系，半结构化，时间序列，地理空间，图像，文本。除了通过相关Python库的经验增强他们的编程技能外，本课程还将向学生介绍使用SQL进行声明性数据处理的概念，以及分析关系数据库中的数据。学生将获得数据集，例如。 ，社交媒体，运输，健康和社会科学，并在小用例的背景下教授基本的探索性数据分析和挖掘技术。该课程将进一步让学生了解分析大量数据所涉及的挑战，例如在多台计算机之间分配和分配数据和计算以处理“大数据”的想法。 本课程介绍是由往届学长学姐回忆编辑而成。 简介：编程语言为Python和SQL，这门课作为Data Science方向的必修课包涵了如何摄取，清理，组合和汇总来自各种数据模型的数据并学习如何视觉化和分析数据，这门课的Quiz是检测SQL的熟练程度，Assignment是利用Python和SQL处理数据。 Tutor简评： 1. 这门课为Double Pass 2. 这门课期末为开卷考试 3. SQL的Quiz很容易拿高分，Assignment给分也不低 4. 期末和老师给的Sample很像，是一个很容易拿高分的课。