Hadoop2.8.0以下版本,内置是不支持按照时间等属性排序的。
我们可以结合Shell命令来实现按照文件最近修改的时间对 ls 命令输出结果进行排序。比如,按照最近修改的时间降序排序(也就是越早修改的文件显示在最下面)
hdfs dfs -ls /spark2-history | sort -r -k6,7
为啥是-k6,7
,因为第6,7个字段是时间字段。
-rwxrwx--- 2 ps_pro hadoop 1809416 2020-12-27 10:22 /spark2-history/application_1608084068201_400785_1
-rwxrwx--- 2 ap_poc hadoop 107590 2020-12-27 10:22 /spark2-history/application_1608084068201_400797_1
-rwxrwx--- 2 fcv hadoop 113705 2020-12-27 10:22 /spark2-history/application_1608084068201_400783_1
-rwxrwx--- 2 datamart_pro hadoop 991453 2020-12-27 10:22 /spark2-history/application_1608084068201_400780_2
-rwxrwx--- 2 datamart_pro hadoop 986122 2020-12-27 10:22 /spark2-history/application_1608084068201_400795_1
-rwxrwx--- 2 datamart_pro hadoop 80683 2020-12-27 10:22 /spark2-history/application_1608084068201_400711_2
-rwxrwx--- 2 rdvs hadoop 163833 2020-12-27 10:21 /spark2-history/application_1608084068201_400779_2
Found 60153 items
效果等同于2.8.0以上的
hdfs dfs -ls -t /spark2-history
Usage: hadoop fs -ls [-d] [-h] [-R] [-t] [-S] [-r] [-u] <args>
Options:
-d: Directories are listed as plain files.
-h: Format file sizes in a human-readable fashion (eg 64.0m instead of 67108864).
-R: Recursively list subdirectories encountered.
-t: Sort output by modification time (most recent first).
-S: Sort output by file size.
-r: Reverse the sort order.
-u: Use access time rather than modification time for display and sorting.
网友评论