Course Overview
Large datasets pose difficulties across the machine learning pipeline. They are difficult to visualize and introduce computational, storage, and communication bottlenecks during data preprocessing and model training. Moreover, high capacity models often used in conjunction with large datasets introduce additional computational and storage hurdles during model training and inference. This course is intended to provide a student with the mathematical, algorithmic, and practical knowledge of issues involving learning with large datasets. Among the topics considered are: data cleaning, visualization, and pre-processing at scale; principles of parallel and distributed computing for machine learning; techniques for scalable deep learning; analysis of programs in terms of memory, computation, and communication complexity; and methods for low-latency inference.
Prerequisites
Students are required to have taken a CMU introductory machine learning course (10-301, 10-315, 10-601, 10-701, or 10-715). A strong background in programming will also be necessary; suggested prerequisites include 15-210, 15-214, or equivalent. Students are expected to be familiar with Python or learn it during the course.
Textbooks
There will be no required textbooks, though we will assign additional reading in the schedule below.
Course Components
The requirements of this course consist of participating in lectures, homework assignments, in-class quizzes, two in-class exams, and a mini-project. The grading breakdown is:
- 20% Exam 1
- 20% Exam 2
- 30% Homework (5 Assignments, 6% Each)
- 10% Writing Sessions
- 15% Project or Mini-Project
- 5% Quizzes on Lecture Content
- Most lectures will be combined with a short quiz which should be taken within 48 hours.
Writing Sessions
As part of each homework, there will be in-class and online "writing sessions" that cover the main ideas from the homework. These writing sessions will be functionally similar to quizzes.
However, they will focus exclusively on homework content. They are designed to be (hopefully) easy if you have completed the homework as intended, and (hopefully) difficult or impossible if you have simply asked
ChatGPT / Claude / Gemini / your classmates to do your homework for you.
Exams
You are required to attend all exams in
person and all quizzes in person, in class. Please
plan your travel accordingly as we will not be able
accommodate individual travel needs (e.g. by offering
the exam early) or conflicts (e.g. enrolling in a
second course).
Homework
The homeworks will be divided into two components:
programming and written. The programming assignments
may ask you to implement ML algorithms from scratch;
they emphasize understanding of real-world
applications of ML, building end-to-end systems, and
experimental design. The written assignments will
focus on core concepts, “on-paper” implementations of
classic learning algorithms, derivations, and
understanding of theory.
Human Work (Slot A) and AI-Assisted Work (Slot B)
The most important learning in this course comes from working
through challenging homework problems yourself. Each homework is
therefore completed in two distinct passes: first, human work that
establishes your own understanding; then, after receiving feedback,
a second pass in which you may use AI and collaborate to correct your
mistakes.
Every homework has two deadlines:
- Slot A (human work only; limited collaboration):
Submit only your own human-generated work. AI assistance is not
permitted. We will grade the submission and identify the questions
you answered incorrectly.
- Slot B (AI-assisted work; full collaboration
permitted): Submit corrections after receiving feedback on
Slot A. You may use AI tools and collaborate with other students.
Only questions that were incorrect in Slot A will be graded in
Slot B.
- For each homework problem, your score is the higher of your Slot
A and Slot B scores. The homework total is 90% of the sum of those
per-problem maximum scores, plus 10% if you earned more than 50% of
the available points in Slot A.
- Submitting AI-assisted work to Slot A, or representing it as
human-only work, is an Academic Integrity Violation and may result
in failure in the course.
- Course staff will offer office hours and answer homework
questions only while Slot A is open.
- Grace days may not be used for Slot A.
As a rule, we do not release PDF solutions for homework. Correct
answers will instead be included in the Gradescope rubric.
Project
Students will create groups and participate in a course project or mini-project. Project details will be released later in the semester.
Quizzes
Participation will be measured via in-class
quizzes. Quizzes can be completed up to 48 hours after the end of
class. Your lowest two quiz scores in the course will be dropped
at the end of the semester. We will not provide make-up quizzes
for days that you miss class.
If you enroll in the course
after a quiz has been released, you will be excused from that
quiz -- it will not count towards or against your final
grade.
Piazza
We will use Piazza for class discussions.
Please go to this Piazza website to join the course forum (note: you must use a cmu.edu email account to join).
We strongly encourage students to post on this forum rather than emailing the course staff directly (this will be more efficient for both students and staff). Students should use Piazza to:
- Ask clarifying questions about the course material.
- Share useful resources with classmates (so long as they do not
contain homework solutions).
- Look for students to form study groups.
- Answer questions posted by other students to solidify your own
understanding of the material.
The course Academic Integrity Policy must be followed on the message boards at all times.
Do not post or request homework solutions! Also, please be polite.
Gradescope
We use
Gradescope to collect PDF submissions of open-ended questions on the homework (e.g. mathematical derivations, plots, short answers). The course staff will manually grade your submission, and you’ll receive personalized feedback explaining your final marks.
You will also submit your code for programming questions on the homework to Gradescope. After uploading your code, our grading scripts will autograde your assignment by running your program on a VM. This provides you with immediate feedback on the performance of your submission.
Regrade Requests
If you believe an error was made during manual grading, you’ll be able to submit a regrade request on Gradescope. For each homework, regrade requests will be open for only **1 week** after the slot B grades have been published. This is to encourage you to check the feedback you’ve received early!
Course Staff
Instructional Staff
Jacob Rast
Teaching Assistants
Aditya Gupta
Schedule
Schedule for Fall 2026 (subject to change)
General Policies
Late Homework Policy
Late Slot A submissions will not be accepted. The late
homework policy below applies only to Slot B submissions.
You have 4 total grace days that can be used to submit late Slot B work without penalty. We will automatically keep a tally of these grace days for you; they will be applied greedily.
You may not use more than 2 grace days on any single homework assignment. Additionally, please note:
- All homework submissions are electronic. As such, lateness will be determined by the latest timestamp of any part of your submission. For example, suppose the homework requires submissions to both Gradescope Written and Programming. If you submit your Written on time but your Programming 1 minute late, your entire homework will be penalized for the full 24-hour period.
- Once you have exhuasted your late days any submission up to 24 hours late will recieve a 50% penalty. Any work submitted after 24 hours will be graded but will not be eligible for any credit.
Extensions
In general, we do not grant extensions on assignments. There are several exceptions:
- Medical Emergencies: If you are sick and unable to complete an assignment or attend class, please go to University Health Services. For minor illnesses, we expect grace days to provide sufficient accommodation. For medical emergencies (e.g. prolonged hospitalization), students may request an extension afterwards by contacting their Student Liaison or Academic Advisor and having them reach out to the education associate Jacob Rast on their behalf.
- Family/Personal Emergencies: If you have a family emergency (e.g. death in the family) or a personal emergency (e.g. mental health crisis), please contact your academic adviser or Counseling and Psychological Services (CaPS). In addition to offering support, they will reach out to the instructors for all your courses on your behalf to request an extension.
- University-Approved Absences: If you are attending an out-of-town university approved event (e.g. multi-day athletic/academic trip organized by the university), you may request an extension for the duration of the trip. You must provide confirmation of your attendance, usually from a faculty or staff organizer of the event.
For any of the above situations, you may request an extension by emailing Jacob Rast (jrast@andrew.cmu.edu). The email should be sent as soon as you are aware of the conflict and at least 5 days prior to the deadline. In the case of an emergency, no notice is needed.
Audit Policy
Official auditing of the course (i.e. taking the course for an “Audit” grade) is not permitted this semester.
Unofficial auditing of the course (i.e. watching the lectures online or attending them in person) is welcome and permitted without prior approval. Unofficial auditors will not be given access to course materials such as homework assignments and exams.
Pass/Fail Policy
Pass/Fail is allowed in this class, no permission is required from the course staff. The grade for the Pass cutoff will depend on your program. Be sure to check with your program / department as to whether you can count a Pass/Fail course towards your degree requirements.
Accommodations for Students with Disabilities
If you have a disability and have an accommodations letter from the Disability Resources office, I encourage you to discuss your accommodations and needs with Jacob Rast as early in the semester as possible. I will work with you to ensure that accommodations are provided as appropriate. If you suspect that you may have a disability and would benefit from accommodations but are not yet registered with the Office of Disability Resources, I encourage you to contact them at
access@andrew.cmu.edu.
Collaboration, AI Use, and Academic Integrity
This section adopted from Matt Gormley's S26 10-301/601 course
Read this section carefully. These rules are intended
to support learning while making the expectations for each submission
slot unambiguous.
Overview: Slot A and Slot B
Each homework has two submissions with different collaboration rules:
- Slot A (Human Work Only): Individual work with
limited collaboration and no AI assistance on homework problems.
- Slot B (AI-Assisted and Collaborative Work): AI
assistance and full collaboration are permitted to correct mistakes
identified in Slot A.
The rules below apply differently to the two slots. Violating the Slot
A rules is treated as an Academic Integrity Violation.
Slot A: Human Work Only (Limited Collaboration, No AI)
General Principles
- Slot A is where the primary learning happens.
- You must submit your own human-generated work.
- You may not use generative AI tools to complete homework problems,
including ChatGPT, GitHub Copilot, Gemini, Claude, Cursor, or code
agents.
- Collaboration should support learning rather than circumvent it.
You may discuss homework problems with classmates only as described
below.
AI Use in Slot A
Allowed:
- You may ask an AI tool high-level questions to understand course
concepts, but you may not ask it to answer or work through a homework
problem. Because the boundary can be difficult to judge, we strongly
recommend that you not use AI tools while Slot A is open. This even extends to
Google AI Search (we recommend you use the "-AI" keyword in your search queries to
disable AI-generated answers) and GitHub Copilot or similar tools in your IDE.
Not allowed:
- Asking an AI tool for an answer to a homework problem.
- Using an IDE with an AI assistant enabled while working on the
homework.
- Using a code agent to complete, modify, or debug homework work.
Collaboration on Written Problems in Slot A
You may discuss ideas, but you may not exchange solutions.
Allowed:
- Discussing concepts and approaches at a whiteboard or chalkboard,
including reasoning through a solution together.
- Taking your own notes after the discussion has ended.
Not allowed:
- Showing or sharing a complete written or electronic solution.
- Copying another student's derivation or final answer.
- Copying shared solutions or work from a whiteboard or chalkboard.
Collaboration on Programming Problems in Slot A
You may discuss code, but you may not copy it.
Allowed:
- Debugging or design discussions around one open laptop.
- High-level discussion of program logic, structure, or algorithmic
ideas.
Not allowed:
- Working with two laptops open side by side.
- Copying, transcribing, or closely paraphrasing another student's
code.
- Taking notes while viewing another student's code.
Disclosure Requirement for Slot A
- List every collaborator for each problem in the homework's
collaboration section.
- Failure to disclose collaboration is itself a violation.
- Collaboration without full disclosure will be handled severely, in
compliance with CMU's Policy on Academic Integrity.
Slot B: AI-Assisted Work and Full Collaboration Permitted
Slot B gives you an opportunity to fix mistakes and deepen your
understanding after receiving feedback on Slot A.
Allowed in Slot B:
- Using generative AI tools for code, explanations, or debugging.
- Working with other students in any format, including shared code
and pair programming.
- Comparing solutions directly.
Restrictions:
- Only questions missed in Slot A are graded in Slot B.
- Slot B work must be submitted to Slot B, not Slot A.
Important: Submitting AI-assisted work to Slot A, or
claiming that AI-assisted work is human-only work, is an Academic
Integrity Violation and may result in severe penalties, including failure
in the course.
Timing of Slot A and Slot B Work
- Do Slot A-style work only during the Slot A submission period.
- Do Slot B-style work only after the Slot A deadline.
These timing rules depend on your self-discipline. If you notice a peer
starting Slot B-style work early, remind them of the policy and ask them to
follow it.
Found Code, Prior Solutions, and External Sources
You may read textbooks, lecture notes, and instructional materials to
understand course concepts.
- During Slot A, write all code and solutions from scratch. Do not use
external code, online or prior solutions, or AI-generated code.
- During Slot B, external code and AI assistance are permitted.
- If you encounter code relevant to an assignment during Slot A,
disclose it in your collaboration statement.
Generative AI tools may be used as resources or collaborators on the
course project unless the project instructions say otherwise.
Duty to Protect One’s Work
Students are responsible for proactively protecting their work from copying and misuse by other students. If a student’s work is copied by another student, the original author is also considered to be at fault and in gross violation of the course policies. It does not matter whether the author allowed the work to be copied or was merely negligent in preventing it from being copied. When overlapping work is submitted by different students, both students will be punished.
To protect future students, do not post your solutions publicly, neither during the course nor afterwards.
Penalties for Violations of Course Policies
All violations (even first one) of course policies will always be reported to the university authorities (your Department Head, Associate Dean, Dean of Student Affairs, etc.) as an official Academic Integrity Violation and will carry severe penalties. The penalty which will be recommended by the Professor for violation of the academic integrity policy is failure in the course. For repeat offenders, violoation of academic integrity policies can even lead to dismissal from the university.
Course Calendar
Objectives
- Given a large scale machine learning task, such as large-scale training or large-scale data preparation, predict which
methods and platforms will be most suitable.
- Implement non-trivial workflows using current scalable/parallel ML platforms.
- Analyze the time, compute, and space complexity of methods for large-scale machine learning.
- Efficiently review recent results in machine learning and critically evaluate them.
- Present experimental results and other technical material clearly in written form.
Acknowledgments
This course is based in part on material developed by Virginia Smith, Ameet Talwalkar, Geoffrey Gordon, Heather Miller, Barnabas Poczos, and Anthony Joseph.
Previous courses:
10-605/10-805, Fall 2025;
10-405/10-605, Spring 2025;
10-605/10-805, Fall 2024;
10-405/10-605, Spring 2024;
10-605/10-805, Fall 2023;
10-405/10-605, Spring 2023;
10-605/10-805, Fall 2022;
10-405/10-605, Spring 2022;
10-605/10-805, Fall 2021;
10-405/10-605, Spring 2021;
10-605/10-805, Fall 2020;
10-405/10-605, Spring 2020;
even older versions of 10-405/10-605/10-805.
