Using Enterprise edition, I first tried to move duplicate iTunes files on my remote drive. NoClone found all the dups and smart mark did a good job in selecting duplicates. When I tried to move the duplicates to a temp folder, the system said it could not locate a certain file. I had to click on that screen, then another screen popped up with "File error" and it needed a click on the ok button. This happened multiple times. I gave up using NoClone on iTunes thinking it was something with iTunes. Total data set size was about 2,400 original files using 10G. At the time I started, there were 10,000 files and 40G. I fixed it without NoClone.
Today I ran NoClone on my photos. 6.5 hours of runtime for 58,000 files in 80G. NoClone found the duplicates (44,000 as a result of the scan) and Smart Mark worked just fine. Then when I tried to move the marked files to a temp folder (on the same remote drive with the name "Throw Me Away") I got a number of the same error messages. First "cannot locate the file" followed by "File Error". Both require manual interaction. This happened mutiple times so I was never able to complete the transfer. In fact, NoClone then crashed so I lost the 6.5 hours of work and I am now rerunning the search.
This work is being done on a Windows XP machine with a networked remote HD. While I am doing this, I have made sure that no other software is running on all my home computers (4 of them) that would access the photos file on the remote drive. So, nothing should be causing the file names to change between the time the scan is done and the time NoClone is to move the found duplicate files.
Help please!
The overnite run took 5:15 because I set a lower limit of 1M. There are roughly half the number of files as I mentioned previously. Again, I am trying to transfer the identified duplicates out of my photos folder on the remote drive and put them in a temp folder on the same drive. I continue to get these error messages. See the attachment for examples.
I did go to the column in the report called Report-read errors, selected all 126 entries and then selected "Ignore" which I hope would cause the software to ignore these errors.
As I write this the system is "moving" the files to the temp folder and I have gotten ~6 of the error messages shown in the attachment.
NOTE:Win2k/XP has MOVE command among its commands, but older Windows platforms (9x, ME, NT) don't have a single (file) MOVE command! In those platforms, you might have to use COPY and DEL commands in a two step operation. The first step would be a batch file that performs COPY operation for each file that needs to be moved to a specific folder, then in the second step, another batch file performs the DEL operation for each file (i.e. delete them from their original folders) that were previously copied to the specific folder. Excel can be used to generate the batch files for either the single-step MOVE operation or the two-step COPY and DEL operations.
IMPORTANT:Select each column separately and format the contents as per data type. In this case, Filename, In Folder columns (columns A and B) should be formatted as TEXT (Format -- Cells -- choose "text"). Columns C, D (Duplicate #, and file Size columns) should be formatted as Integer (Format -- Cells -- choose "number", with no decimal places, and you can select thousand separator). Column E (Date Modified) should be formatted as date (Format -- Cells -- Custom, type: <yyyy-mm-dd hh:mm>). The formatting is important, because it will prevent any mistakes due to data type mismatch during sorting or comparison operations!
=IF(C3=C2,"del","keep")
move [/y | /-y] [source] [target]/y or /-yare switches that prompts you to confirm when you want to overwrite an existing destination file. The default is /y, which means you will be prompted should a destination file exist which a move command might want to overwrite. For our purposes, we don't want to overwrite any file, whether we are prompted to overwrite it or not! Thus, we need to figure out a way to make sure each file being sent/moved to a destination folder is unique! Personally, I prefer to use the original pathname of a file as part of the new filename. (more on this issue later)sourcethe pathname and filename of the file that is going to be movedtargetthe pathname and [new] filename to move the files to.
EX: original file: D:\sample docs\temp\file-01.txt [converted] new filename: d~sample docs~temp~file-01.txt
you can use FIND in Excel to search if the character you've chosen is ever used in the CSV file, since all the files and pathnames you'll be working with are already listed in the CSV file! There's another reason to using a unique character for the purposes of converting a pathname into a regular text string: the same character can be used to parse filenames so that a pathname could be extracted and reconstructed, especially if you want to re-create a folder structure on a different drive, and move the files to the re-created/re-constructed folders and subfolders!
to test if you're exceeding the filename length limits, you can use LEN function and total the contents of Columns A and B contents.
EX:in H2, type in "=Len(A2)+Len(B2)" and copy and paste this formula into the rows under column H. When you reach the end of the data list in column H, use the MAX() function and select the rows above as the argument for MAX() function. MAX() will show you the maximum value encountered among all the data in column H, so you'll know what's the longest length of filename you might end up using with the file naming scheme I've proposed.[see cap-05.gif, cap-06.gif]
According to Win2k/XP help file, a filename can be max 215 characters long (excluding the pathname), including the spaces. It cannot contain \ / : * ? " < > | characters.If it seems like you are going to have a very long filename due to long and deeply nested folders, think of work-arounds. You can use an index (integer) as part of the filename, incremented from row to row, and add this index number to the filename, separating it with the unique character ("~"), such as file-01~1248.txt or 1248~file-01.txt (for a file located in row 1248). As long as you keep the Excel file, you can always reconstruct the folder name, by performing "lookup" operations in Excel using its many lookup functions and the new filename!
Example (using our new file naming convention for the new files): MOVE "D:\test\f04-miss-mid.txt" "C:\temp\d~test~f04-miss-mid.txt"or MOVE "D:\test\f04-miss-mid.txt" "C:\temp\test~f04-miss-mid.txt"
Find what= D:\Replace with= <empty>click on Replace All. [see cap-08.gif]Find what= \Replace with= ~click on Replace All. [see cap-09.gif]
NOTES:I've used some extra <space> characters in some of the terms of CONCATENATE function to obtain a blocked look, but it's not really necessary, and you can get by with only one <space> character.because <path+filename> terms may include spaces, you need to enclose the <source> and <target> items in double quotes, thus, CHAR(34) is used which equals to a double-quote character.
If <comparison is true> then <output MOVE command> <comparison is false> then <output "blank" or "empty">Original IF() function in cell G3 was: =IF(C3=C2,"del","keep")Original CONCATENATE() function in cell I3 was:=CONCATENATE("move ",CHAR(34),B3,A3,CHAR(34)," ",CHAR(34),"C:\temp\",H3,A3,CHAR(34))Thus, in cell G3, we edit the IF() function and substitute the CONCATENATE() function contents in place of the "del" term (i.e. for the case when comparison is true), and in place of the "keep" term (i.e. when the comparison is false) we replace it with an empty input ("" two double-quotes with no space in between). Finally, we copy and paste G3 formula into the rows below until we reach the end of the listed data.The final formula in Cell G3 will be:=IF(C3=C2,CONCATENATE("move ",CHAR(34),B3,A3,CHAR(34)," ",CHAR(34),"C:\temp\",H3,A3,CHAR(34)),"")[see cap-14.gif]
Find what= ^p^pReplace with= ^pand click on Replace All. It will remove all the blank lines.
NOTE:As I said, I never tested moving files of very large size, although I did perform a test with 500 - 1,000 MB size, and this method worked, but... I'm not sure if there's a file size limit that might cause problems. Just make sure your destination drive will have the necessary empty space for the files you want to move to.