Duplicate Sentence Checker and Remover

Task:

Create a program that can read a text file and check for duplicate sentences. If a sentence is found to be duplicated, it should be removed from the text file. The program should then save the updated text file with the removed duplicate sentences.

Solution:

To complete this task, we will be using the following steps:

• Import necessary modules: We will be using the "re" module to perform regular expression operations and the "os" module to interact with the operating system.
• Open the text file: We will use the "open()" function to open the text file in read mode and assign it to a variable.
• Read the file: We will use the "read()" function to read the contents of the file and assign it to a variable.
• Create a regex pattern: We will use a regular expression pattern to match sentences in the text file. The pattern will search for any combination of words, numbers, and punctuation marks followed by a period and a space.
• Find duplicate sentences: We will use the "findall()" function from the "re" module to find all the duplicate sentences in the text file based on the regex pattern. The function will return a list of all the duplicate sentences found.
• Remove duplicate sentences: We will use the "replace()" function to remove all the duplicate sentences found in the text file and assign the updated text to a variable.
• Save the updated text file: We will use the "write()" function to save the updated text to a new text file. This will ensure that the original text file remains unchanged.
• Close the file: Once the program is done, we will close the file using the "close()" function.
• Test the program: We will test the program by running it with different text files and checking the output to ensure that duplicate sentences are removed successfully. 
• Add error handling: We will add appropriate error handling to handle any exceptions that may occur during the execution of the program.
• Optional: We can also add a feature to prompt the user to enter the name of the text file they want to check for duplicate sentences, rather than hard-coding the file name in the program.

Overall, the program will be able to efficiently check for duplicate sentences in a text file and remove them, thereby helping to improve the readability and accuracy of the text.

Duplicate Sentence Checker and Remover

Enter text: Output: