{ "cells": [ { "cell_type": "markdown", "id": "cell-23-22-00", "metadata": {}, "source": [ "# 23-22 · Тесты дубликатов и пустых файлов\n", "\n", "Практика к разделу [«Проверяем поиск дубликатов»](../../site/chapters/glava-23/23-26-testy-dublikatov.html). Повторяет `projects/python/safesort/tests/test_duplicates.py`, но полностью в памяти — без обращения к диску." ] }, { "cell_type": "markdown", "id": "cell-23-22-01", "metadata": {}, "source": [ "## Цель\n", "\n", "Написать и запустить тесты для группировки дубликатов: типичный случай, файлы без пары и — отдельно — пустые файлы, которые тоже считаются дубликатами друг друга." ] }, { "cell_type": "markdown", "id": "cell-23-22-02", "metadata": {}, "source": [ "## Example" ] }, { "cell_type": "code", "execution_count": null, "id": "cell-23-22-03", "metadata": {}, "outputs": [], "source": [ "import hashlib\n", "from collections import defaultdict\n", "from dataclasses import dataclass\n", "\n", "\n", "@dataclass(frozen=True)\n", "class FileInfo:\n", " name: str\n", " size: int\n", " content: bytes\n", "\n", "\n", "def find_duplicates(files):\n", " by_size = defaultdict(list)\n", " for file in files:\n", " by_size[file.size].append(file)\n", "\n", " groups = []\n", " for size, candidates in by_size.items():\n", " if len(candidates) < 2:\n", " continue\n", " by_digest = defaultdict(list)\n", " for candidate in candidates:\n", " digest = hashlib.sha256(candidate.content).hexdigest()\n", " by_digest[digest].append(candidate)\n", " for digest, matched in by_digest.items():\n", " if len(matched) >= 2:\n", " groups.append({\"size\": size, \"digest\": digest, \"files\": tuple(matched)})\n", " return groups\n", "\n", "\n", "def test_identical_content_files_are_grouped():\n", " files = [\n", " FileInfo(\"notes.txt\", 12, b\"tot zhe text\"),\n", " FileInfo(\"copy_of_notes.txt\", 12, b\"tot zhe text\"),\n", " ]\n", " groups = find_duplicates(files)\n", " assert len(groups) == 1\n", " assert len(groups[0][\"files\"]) == 2\n", "\n", "\n", "def test_different_content_same_size_not_grouped():\n", " files = [\n", " FileInfo(\"a.txt\", 4, b\"AAAA\"),\n", " FileInfo(\"b.txt\", 4, b\"BBBB\"),\n", " ]\n", " groups = find_duplicates(files)\n", " assert groups == []\n", "\n", "\n", "def test_empty_files_are_duplicates_of_each_other():\n", " files = [\n", " FileInfo(\"a.txt\", 0, b\"\"),\n", " FileInfo(\"b.txt\", 0, b\"\"),\n", " ]\n", " groups = find_duplicates(files)\n", " assert len(groups) == 1\n", " assert groups[0][\"size\"] == 0\n", " assert groups[0][\"digest\"] == hashlib.sha256(b\"\").hexdigest()\n", "\n", "\n", "for test_func in (\n", " test_identical_content_files_are_grouped,\n", " test_different_content_same_size_not_grouped,\n", " test_empty_files_are_duplicates_of_each_other,\n", "):\n", " test_func()\n", " print(f\"OK: {test_func.__name__}\")" ] }, { "cell_type": "markdown", "id": "cell-23-22-04", "metadata": {}, "source": [ "## Проверка результата" ] }, { "cell_type": "code", "execution_count": null, "id": "cell-23-22-05", "metadata": {}, "outputs": [], "source": [ "groups_dlya_proverki = find_duplicates([\n", " FileInfo(\"x1.bin\", 3, b\"XXX\"),\n", " FileInfo(\"x2.bin\", 3, b\"XXX\"),\n", " FileInfo(\"x3.bin\", 3, b\"XXX\"),\n", "])\n", "\n", "assert len(groups_dlya_proverki) == 1\n", "assert len(groups_dlya_proverki[0][\"files\"]) == 3\n", "print(\"Верно: три файла с одинаковым содержимым образовали одну группу из трёх, \"\n", " \"а не полтора дубликата.\")" ] }, { "cell_type": "markdown", "id": "cell-23-22-06", "metadata": {}, "source": [ "## Starter\n", "\n", "Заполните отмеченное место. Неизменённый starter не проходит tests." ] }, { "cell_type": "code", "execution_count": null, "id": "task-23-22", "metadata": { "tags": [ "exercise", "starter" ] }, "outputs": [], "source": [ "def imena_duplicate_groups(files):\n", " # TODO: call find_duplicates and return a set of frozenset names.\n", " raise NotImplementedError\n", "\n", "\n", "imena_grupp = imena_duplicate_groups([\n", " FileInfo(\"a1.txt\", 5, b\"AAAAA\"),\n", " FileInfo(\"a2.txt\", 5, b\"AAAAA\"),\n", " FileInfo(\"b1.txt\", 7, b\"BBBBBBB\"),\n", " FileInfo(\"c1.txt\", 9, b\"CCCCCCCCC\"),\n", "])" ] }, { "cell_type": "markdown", "id": "cell-23-22-08", "metadata": {}, "source": [ "## Task\n", "\n", "Верните множества имён для всех найденных duplicate groups. Уникальные размеры не должны появиться." ] }, { "cell_type": "markdown", "id": "cell-23-22-09", "metadata": {}, "source": [ "## Tests\n", "\n", "Запустите после task cell: есть основной пример и хотя бы один крайний случай." ] }, { "cell_type": "code", "execution_count": null, "id": "tests-23-22", "metadata": { "tags": [ "exercise-tests" ] }, "outputs": [], "source": [ "assert imena_grupp == {frozenset({\"a1.txt\", \"a2.txt\"})}\n", "assert imena_duplicate_groups([]) == set()\n", "assert imena_duplicate_groups([FileInfo(\"x\", 1, b\"x\")]) == set()\n", "print(\"Tests passed\")" ] }, { "cell_type": "markdown", "id": "cell-23-22-11", "metadata": {}, "source": [ "## Hint\n", "\n", "Преобразуйте каждую `group[\"files\"]` во `frozenset(file.name ...)`." ] }, { "cell_type": "markdown", "id": "cell-23-22-12", "metadata": {}, "source": [ "## Solution\n", "\n", "
Показать решение после собственной попытки\n", "\n", "```python\n", "def imena_duplicate_groups(files):\n", " return {\n", " frozenset(file.name for file in group[\"files\"])\n", " for group in find_duplicates(files)\n", " }\n", "\n", "\n", "imena_grupp = imena_duplicate_groups([\n", " FileInfo(\"a1.txt\", 5, b\"AAAAA\"),\n", " FileInfo(\"a2.txt\", 5, b\"AAAAA\"),\n", " FileInfo(\"b1.txt\", 7, b\"BBBBBBB\"),\n", " FileInfo(\"c1.txt\", 9, b\"CCCCCCCCC\"),\n", "])\n", "```\n", "\n", "
" ] } ], "metadata": { "kernelspec": { "display_name": "Cartesian Python 3.14", "language": "python", "name": "cartesian-python314" }, "language_info": { "name": "python", "version": "3.14.6" } }, "nbformat": 4, "nbformat_minor": 5 }