php删除文本文件中重复行的方法

本文实例讲述了php删除文本文件中重复行的方法。分享给大家供大家参考。具体分析如下：

这个php函数用来删除文件中的重复行，还可以指定是否忽略大小写，和指定换行符

/** * RemoveDuplicatedLines * This function removes all duplicated lines of the given text file. * * @param   string * @param   bool * @return  string */function RemoveDuplicatedLines($Filepath, $IgnoreCase=false, $NewLine="/n"){  if (!file_exists($Filepath)){    $ErrorMsg = 'RemoveDuplicatedLines error: ';    $ErrorMsg .= 'The given file ' . $Filepath . ' does not exist!';    die($ErrorMsg);  }  $Content = file_get_contents($Filepath);  $Content = RemoveDuplicatedLinesByString($Content, $IgnoreCase, $NewLine);  // Is the file writeable?  if (!is_writeable($Filepath)){    $ErrorMsg = 'RemoveDuplicatedLines error: ';    $ErrorMsg .= 'The given file ' . $Filepath . ' is not writeable!';      die($ErrorMsg);  }  // Write the new file  $FileResource = fopen($Filepath, 'w+');     fwrite($FileResource, $Content);      fclose($FileResource);  } /** * RemoveDuplicatedLinesByString * This function removes all duplicated lines of the given string. * * @param   string * @param   bool * @return  string */function RemoveDuplicatedLinesByString($Lines, $IgnoreCase=false, $NewLine="/n"){  if (is_array($Lines))    $Lines = implode($NewLine, $Lines);  $Lines = explode($NewLine, $Lines);  $LineArray = array();  $Duplicates = 0;  // Go trough all lines of the given file  for ($Line=0; $Line < count($Lines); $Line++){    // Trim whitespace for the current line    $CurrentLine = trim($Lines[$Line]);    // Skip empty lines    if ($CurrentLine == '')      continue;    // Use the line contents as array key    $LineKey = $CurrentLine;    if ($IgnoreCase)      $LineKey = strtolower($LineKey);    // Check if the array key already exists,    // if not add it otherwise increase the counter    if (!isset($LineArray[$LineKey]))      $LineArray[$LineKey] = $CurrentLine;        else              $Duplicates++;  }  // Sort the array  asort($LineArray);  // Return how many lines got removed  return implode($NewLine, array_values($LineArray));  }

使用范例：

// Example 1// Removes all duplicated lines of the file definied in the first parameter.$RemovedLinesCount = RemoveDuplicatedLines('test.txt');print "Removed $RemovedLinesCount duplicate lines from the test.txt file.";// Example 2 (Ignore case)// Same as above, just ignores the line case.RemoveDuplicatedLines('test.txt', true);// Example 3 (Custom new line character)// By using the 3rd parameter you can define which character// should be used as new line indicator. In this case// the example file looks like 'foo;bar;foo;foo' and will// be replaced with 'foo;bar' RemoveDuplicatedLines('test.txt', false, ';');

希望本文所述对大家的php程序设计有所帮助。