en:tools:linux:file-handling [Wiki]

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
==== File sort using extensions ====
Moves files to folders based on their extension : 
<code bash>
#!/bin/sh

cd ~/Downloads
pwd

## Image
echo "+ moving images"

find . -maxdepth 1 -type f \
-name "*jpg" -o -name "*JPG" \
-o -name "*jpeg" -o -name "*JPEG" \
-o -name "*png" -o -name "*PNG" \
| xargs -I '{}' mv {} images


## Document
echo "+ moving documents"

find . -maxdepth 1 -type f \
-name "*pdf" -o -name "*PDF" \
-o -name "*.doc" -o -name "*.DOC" \
-o -name "*.docx" -o -name "*.DOCX" \
-o -name "*.ppt" -o -name "*.PPT" \
-o -name "*.pptx" -o -name "*.PPTX" \
-o -name "*.xls" -o -name "*.XLS" \
-o -name "*.xlsx" -o -name "*.XLSX" \
-o -name "*odt" -o -name "*ODT" \
-o -name "*ods" -o -name "*ODS" \
-o -name "*odp" -o -name "*ODP" \
| xargs -I '{}' mv {} documents


## CAO
echo "+ moving CAO files"

find . -maxdepth 1 -type f \
-name "*stl" -o -name "*STL" \
-o -name "*dwg" -o -name "*DWG" \
-o -name "*dxf" -o -name "*DXF" \
| xargs -I '{}' mv {} CAO


## Vidéo
echo "+ moving videos"

find . -maxdepth 1 -type f \
-name "*mov" -o -name "*MOV" \
-o -name "*mp4" -o -name "*MP4" \
-o -name "*avi" -o -name "*AVI" \
-o -name "*mkv" -o -name "*MKV" \
| xargs -I '{}' mv {} videos


## Div execs / data / archives
echo "+ moving the rest"

find . -maxdepth 1 -type f \
-name "*txt" -o -name "*TXT" \
-o -name "*iso" -o -name "*ISO" \
-o -name "*csv" -o -name "*CSV" \
-o -name "*ics" -o -name "*ICS" \
-o -name "*sh" -o -name "*SH" \
-o -name "*py" -o -name "*PY" \
-o -name "*exe" -o -name "*EXE" \
-o -name "*AppImage" \
-o -name "*msi" -o -name "*MSI" \
-o -name "*deb" -o -name "*DEB" \
-o -name "*rpm" -o -name "*RPM" \
-o -name "*zip" -o -name "*ZIP" \
-o -name "*rar" -o -name "*RAR" \
-o -name "*.tar" -o -name "*.TAR" \
| xargs -I '{}' mv {} div-data-exe-archives

## remove ext
echo "- removing exts"
rm *.ext 2>/dev/null

## End information
echo ""
echo "unsorted files"
ls -al | grep '^-' | wc -l
echo ""
echo "Size on disk "
du -sh
echo ""

echo "done"
</code>

==== copy while preserving arborescence ====
[[https://ostechnix.com/copy-specific-file-types-while-keeping-directory-structure-in-linux/|source]]

<code bash>find. -name '*.mp3' -exec cp --parents \{\}~/target\;</code>

   * ''find'' – command to find files and folders in Unix-like systems.
   * The dot (''.'') - represents that we copied the contents from current directory.
   * ''-name ‘*.mp3''' – search for files with the extension .mp3
   * ''-exec cp'' – execute the ‘cp’ command to copy files from source to destination directory.
   * ''--parents'' - create the intermediate parent directories if needed to preserve the parent directory structure.
   * ''\{\}'' – is automatically replaced with the file name of the files found by the ‘find’ command. And the braces are escaped to protect them from expansion by the shell in some "find" command versions. You can also use ''{}'' without escape characters.
   * ''~/target'' – target directory to save the matching files.
   * ''\;'' – indicates it that the commands to be executed are now complete, and to carry out the command again on the next match.

==== recursively copy all files without preserving the file tree ====

<code bash>find. -name '*.PNG' -exec cp\{\}allpics/\;</code>

Here we are at the root of the file tree to explore and the destination folder is ''allpics''.

==== list the differences between two directories ====
count the differences
<code bash>diff -y <(cd /path/to/files/1/ && ls -R) <(cd /path/to/files/2/ && ls -R) | wc -l</code>

list the differences in diff.log
<code bash>diff -y <(cd /path/to/files/1/ && ls -R) <(cd /path/to/files/2/ && ls -R) > diff.log</code>

adding ''grep -E \/'' or ''-d'' on the ''ls'' allows you to list directories

==== list the duplicates ====
[[https://buildvirtual.net/how-to-find-duplicate-files-on-linux/|source]]

**detect duplicates**

<code bash>fdupes -Sr .</code>

==== Disk usage ====
Usefull commands : 
<code bash>
#liste the 30 largest files in filetree
du -Sh | sort -rh | head -30</code>
<code bash>
#ncurse disk usage
ncdu
#on remote system
ssh -C user@system ncdu -o- / | ./ncdu -f-
</code>