CTF

Magnet CTF Week 6: ELFant hunting

 · 19 mins read

TL;DR: Week 6 of the #MagnetWeeklyCTF was the beginning of what I can only hope is a good reverse engineering side quest.

Review

Check out the week 1 blog post for how to get started on the Magnet Weekly CTF.

Get the first challenge

The weekly challenge for week 6 was two parts, both lengthy. The first was:

Hadoop is a complex framework from Apache used to perform distributed processing of large data sets. Like most frameworks, it relies on many dependencies to run smoothly. Fortunately, it’s designed to install all of these dependencies automatically. On the secondary nodes (not the MAIN node) your colleague recollects seeing one particular dependency failed to install correctly. Your task is to find the specific error code that led to this failed dependency installation. [Flag is numeric]

This is getting back into things you learn running Linux as your daily driver for over 15 years, unlike Hadoop questions. The host is running Ubuntu, so we would expect to see the user running apt to install and manage dependencies.

Open the target file(s)

The log files relating to apt can be found in /var/log/apt. In /var/log/apt/history.log you can see the commands run to manage packages, so this was the first stop to find a dependency which did not install. Comparing the three disks, HDFS-Slave2 had errors worth digging in to.

[notta@cuppa apt]$ cat history.log

...[snip]...

Start-Date: 2017-11-07  23:17:44
Commandline: apt-get -o APT::Status-Fd=4 -o APT::Keep-Fds::=5 -o APT::Keep-Fds::=6 -q -y --no-remove install grub-pc
Install: grub2-common:amd64 (2.02~beta2-36ubuntu3.14, automatic), grub-pc:amd64 (2.02~beta2-36ubuntu3.14), grub-pc-bin:amd64 (2.02~beta2-36ubuntu3.14, automatic), grub-gfxpayload-lists:amd64 (0.7, automatic)
End-Date: 2017-11-07  23:17:45

Start-Date: 2017-11-08  01:17:04
Commandline: apt-get install oracle-java7-installer
Requested-By: hadoop (1000)
Install: libfontenc1:amd64 (1:1.1.3-1, automatic), binutils:amd64 (2.26.1-1ubuntu1~16.04.5, automatic), xfonts-utils:amd64 (1:7.7+3ubuntu0.16.04.2, automatic), oracle-java7-installer:amd64 (7u80+7u60arm-0~webupd8~1), gsfonts-x11:amd64 (0.24, automatic), libxfont1:amd64 (1:1.5.1-1ubuntu0.16.04.3, automatic), gsfonts:amd64 (1:8.11+urwcyr1.0.7~pre44-4.2ubuntu1, automatic), x11-common:amd64 (1:7.7+13ubuntu3, automatic), xfonts-encodings:amd64 (1:1.0.4-2, automatic), java-common:amd64 (0.56ubuntu2, automatic)
Error: Sub-process /usr/bin/dpkg returned an error code (1)
End-Date: 2017-11-08  01:20:06

Start-Date: 2017-11-08  01:23:15
Commandline: apt-get install oracle-java8-installer
Requested-By: hadoop (1000)
Install: oracle-java8-set-default:amd64 (8u151-1~webupd8~0, automatic), oracle-java8-installer:amd64 (8u151-1~webupd8~0)
Error: Sub-process /usr/bin/dpkg returned an error code (1)
End-Date: 2017-11-08  01:28:31

...[snip]...

The first error in that file came with a command started at 01:17:04 on November 8, 2017, but we can’t immediately see the error. However, we can examine term.log to see what the user was actually presented with at that time.

[notta@cuppa apt]$ cat history.log

...[snip]...

Unpacking oracle-java7-installer (7u80+7u60arm-0~webupd8~1) ...
Selecting previously unselected package gsfonts.
Preparing to unpack .../gsfonts_1%3a8.11+urwcyr1.0.7~pre44-4.2ubuntu1_all.deb ...
Unpacking gsfonts (1:8.11+urwcyr1.0.7~pre44-4.2ubuntu1) ...
Selecting previously unselected package libfontenc1:amd64.
Preparing to unpack .../libfontenc1_1%3a1.1.3-1_amd64.deb ...
Unpacking libfontenc1:amd64 (1:1.1.3-1) ...
Selecting previously unselected package libxfont1:amd64.
Preparing to unpack .../libxfont1_1%3a1.5.1-1ubuntu0.16.04.3_amd64.deb ...
Unpacking libxfont1:amd64 (1:1.5.1-1ubuntu0.16.04.3) ...
Selecting previously unselected package x11-common.
Preparing to unpack .../x11-common_1%3a7.7+13ubuntu3_all.deb ...
Unpacking x11-common (1:7.7+13ubuntu3) ...
Selecting previously unselected package xfonts-encodings.
Preparing to unpack .../xfonts-encodings_1%3a1.0.4-2_all.deb ...
Unpacking xfonts-encodings (1:1.0.4-2) ...
Selecting previously unselected package xfonts-utils.
Preparing to unpack .../xfonts-utils_1%3a7.7+3ubuntu0.16.04.2_amd64.deb ...
Unpacking xfonts-utils (1:7.7+3ubuntu0.16.04.2) ...
Selecting previously unselected package gsfonts-x11.
Preparing to unpack .../gsfonts-x11_0.24_all.deb ...
Unpacking gsfonts-x11 (0.24) ...
Processing triggers for mime-support (3.59ubuntu1) ...
Processing triggers for shared-mime-info (1.5-2ubuntu0.1) ...
Processing triggers for libc-bin (2.23-0ubuntu3) ...
Processing triggers for systemd (229-4ubuntu7) ...
Processing triggers for ureadahead (0.100.0-19) ...
Processing triggers for man-db (2.7.5-1) ...
Setting up java-common (0.56ubuntu2) ...
Setting up oracle-java7-installer (7u80+7u60arm-0~webupd8~1) ...
Downloading Oracle Java 7...
--2017-11-08 01:19:56--  http://download.oracle.com/otn-pub/java/jdk/7u80-b15/jdk-7u80-linux-x64.tar.gz
Resolving download.oracle.com (download.oracle.com)... 151.248.100.33, 151.248.100.43
Connecting to download.oracle.com (download.oracle.com)|151.248.100.33|:80... connected.
HTTP request sent, awaiting response... 302 Moved Temporarily
Location: https://edelivery.oracle.com/otn-pub/java/jdk/7u80-b15/jdk-7u80-linux-x64.tar.gz [following]
--2017-11-08 01:19:59--  https://edelivery.oracle.com/otn-pub/java/jdk/7u80-b15/jdk-7u80-linux-x64.tar.gz
Resolving edelivery.oracle.com (edelivery.oracle.com)... 104.117.142.224, 2a02:26f0:c00:48f::2d3e, 2a02:26f0:c00:4bd::2d3e
Connecting to edelivery.oracle.com (edelivery.oracle.com)|104.117.142.224|:443... connected.
HTTP request sent, awaiting response... 302 Moved Temporarily
Location: http://download.oracle.com/otn-pub/java/jdk/7u80-b15/jdk-7u80-linux-x64.tar.gz?AuthParam=1510096924_89bcd8b3c695fffc7000931d0567a6d9 [following]
--2017-11-08 01:20:02--  http://download.oracle.com/otn-pub/java/jdk/7u80-b15/jdk-7u80-linux-x64.tar.gz?AuthParam=1510096924_89bcd8b3c695fffc7000931d0567a6d9
Connecting to download.oracle.com (download.oracle.com)|151.248.100.33|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2017-11-08 01:20:05 ERROR 404: Not Found.

Here we can see the error came from trying to install a Java 7 package and that the error was specifically a 404 HTTP error which prevented its download, probably because that is an ancient package. Since the answer wants the numeric code, “404” is the answer for the first part.

Get the second challenge

The second part was:

Don’t panic about the failed dependency installation. A very closely related dependency was installed successfully at some point, which should do the trick. Where did it land? In that folder, compared to its binary neighbors nearby, this particular file seems rather an ELFant. Using the error code from your first task, search for symbols beginning with the same number (HINT: leading 0’s don’t count). There are three in particular whose name share a common word between them. What is the word?

If “ELF” wasn’t a dead giveaway to you, we can translate it into Windows by saying “EXEant” although the impact isn’t the same. ELF stands for “Executable and Linkable Format” and is the standard format for executables on your normal Linux workstation, similar to EXE on Windows. Right away I was hopeful we would be doing some reverse engineering.

Find the wrong place

The first part of the question forces you to find the dependency that was installed. We already saw in the history.log file that oracle-java8-installer was attempted immediately after the failed dependency and term.log shows that jdk-8u151-linux-x64.tar.gz was downloaded to install. This is the first place I went down the wrong rabbit hole, because that file is found in /home/hadoop/temp/, so I thought that was the location they were talking about.

Find the wrong binary

In the same folder as the zip file is an ELF binary named master on the main node, this is the second place I went wrong because the question clearly states that you did not want the main node. But feel free to join me on my journey down the rabbit hole. This binary, however was stripped, meaning you can’t find the names of its symbols, so it can’t be the right answer, but seems funky since it has netcat in it. Immediately above this folder is a binary named 45010 which also looks fairly suspicious. Running strings on it has phrases such as “credentials patched, launching shell…”, so this has to be the binary we are digging in to, right? We will use objdump to take a look at the symbol table, specically with the -t switch.

[notta@cuppa hadoop]$ file 45010 

45010: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, 
interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=38f8ab3652358f154d8da3a131bfb8b1832ec23d, for GNU/Linux 3.2.0, not stripped

[notta@cuppa hadoop]$ objdump -t 45010 

45010:     file format elf64-x86-64

SYMBOL TABLE:
00000000000002a8 l    d  .interp  0000000000000000              .interp
00000000000002c4 l    d  .note.gnu.build-id 0000000000000000              .note.gnu.build-id
00000000000002e8 l    d  .note.ABI-tag  0000000000000000              .note.ABI-tag
0000000000000308 l    d  .gnu.hash  0000000000000000              .gnu.hash
0000000000000338 l    d  .dynsym  0000000000000000              .dynsym
0000000000000530 l    d  .dynstr  0000000000000000              .dynstr
000000000000061c l    d  .gnu.version 0000000000000000              .gnu.version
0000000000000648 l    d  .gnu.version_r 0000000000000000              .gnu.version_r
0000000000000668 l    d  .rela.dyn  0000000000000000              .rela.dyn
0000000000000758 l    d  .rela.plt  0000000000000000              .rela.plt
0000000000001000 l    d  .init  0000000000000000              .init
0000000000001020 l    d  .plt 0000000000000000              .plt
0000000000001100 l    d  .plt.got 0000000000000000              .plt.got
0000000000001110 l    d  .text  0000000000000000              .text
0000000000002864 l    d  .fini  0000000000000000              .fini
0000000000003000 l    d  .rodata  0000000000000000              .rodata
0000000000003384 l    d  .eh_frame_hdr  0000000000000000              .eh_frame_hdr
0000000000003468 l    d  .eh_frame  0000000000000000              .eh_frame
0000000000004de8 l    d  .init_array  0000000000000000              .init_array
0000000000004df0 l    d  .fini_array  0000000000000000              .fini_array
0000000000004df8 l    d  .dynamic 0000000000000000              .dynamic
0000000000004fd8 l    d  .got 0000000000000000              .got
0000000000005000 l    d  .got.plt 0000000000000000              .got.plt
0000000000005080 l    d  .data  0000000000000000              .data
00000000000050a0 l    d  .bss 0000000000000000              .bss
0000000000000000 l    d  .comment 0000000000000000              .comment
0000000000000000 l    df *ABS*  0000000000000000              crtstuff.c
0000000000001140 l     F .text  0000000000000000              deregister_tm_clones
0000000000001170 l     F .text  0000000000000000              register_tm_clones
00000000000011b0 l     F .text  0000000000000000              __do_global_dtors_aux
00000000000050c8 l     O .bss 0000000000000001              completed.7444
0000000000004df0 l     O .fini_array  0000000000000000              __do_global_dtors_aux_fini_array_entry
00000000000011f0 l     F .text  0000000000000000              frame_dummy
0000000000004de8 l     O .init_array  0000000000000000              __frame_dummy_init_array_entry
0000000000000000 l    df *ABS*  0000000000000000              45010.c
00000000000011f5 l     F .text  000000000000000e              ptr_to_u64
000000000000142d l     F .text  00000000000009da              load_prog
0000000000002339 l     F .text  0000000000000062              writemsg
000000000000239b l     F .text  0000000000000052              update_elem
00000000000023ed l     F .text  000000000000004f              get_value
000000000000243c l     F .text  000000000000005d              sendcmd
0000000000002519 l     F .text  000000000000016e              find_cred
0000000000002687 l     F .text  00000000000000f2              hammer_cred
0000000000000000 l    df *ABS*  0000000000000000              crtstuff.c
000000000000380c l     O .eh_frame  0000000000000000              __FRAME_END__
0000000000000000 l    df *ABS*  0000000000000000              
0000000000004df0 l       .init_array  0000000000000000              __init_array_end
0000000000004df8 l     O .dynamic 0000000000000000              _DYNAMIC
0000000000004de8 l       .init_array  0000000000000000              __init_array_start
0000000000003384 l       .eh_frame_hdr  0000000000000000              __GNU_EH_FRAME_HDR
0000000000005000 l     O .got.plt 0000000000000000              _GLOBAL_OFFSET_TABLE_
0000000000001000 l     F .init  0000000000000000              _init
0000000000002860 g     F .text  0000000000000001              __libc_csu_fini
0000000000000000       F *UND*  0000000000000000              __errno_location@@GLIBC_2.2.5
0000000000000000  w      *UND*  0000000000000000              _ITM_deregisterTMCloneTable
00000000000050a0 g     O .bss 0000000000000008              stdout@@GLIBC_2.2.5
0000000000005080  w      .data  0000000000000000              data_start
0000000000000000       F *UND*  0000000000000000              setsockopt@@GLIBC_2.2.5
0000000000000000       F *UND*  0000000000000000              write@@GLIBC_2.2.5
000000000000208d g     F .text  00000000000000d0              fail
0000000000005090 g       .data  0000000000000000              _edata
0000000000002864 g     F .fini  0000000000000000              .hidden _fini
0000000000000000       F *UND*  0000000000000000              getuid@@GLIBC_2.2.5
00000000000050e0 g     O .bss 0000000000000008              sockets
00000000000050e8 g     O .bss 0000000000000004              mapfd
00000000000050ec g     O .bss 0000000000000004              progfd
000000000000215d g     F .text  00000000000001dc              initialize
0000000000000000       F *UND*  0000000000000000              __libc_start_main@@GLIBC_2.2.5
0000000000005100 g     O .bss 0000000000000040              buffer
0000000000005140 g     O .bss 0000000000010000              bpf_log_buf
0000000000005080 g       .data  0000000000000000              __data_start
0000000000000000       F *UND*  0000000000000000              fprintf@@GLIBC_2.2.5
0000000000000000       F *UND*  0000000000000000              syscall@@GLIBC_2.2.5
00000000000012bf g     F .text  000000000000006c              bpf_create_map
0000000000000000  w      *UND*  0000000000000000              __gmon_start__
0000000000005088 g     O .data  0000000000000000              .hidden __dso_handle
0000000000000000       F *UND*  0000000000000000              socketpair@@GLIBC_2.2.5
0000000000003000 g     O .rodata  0000000000000004              _IO_stdin_used
000000000000132b g     F .text  000000000000008a              bpf_update_elem
0000000000001203 g     F .text  00000000000000bc              bpf_prog_load
0000000000002800 g     F .text  000000000000005d              __libc_csu_init
0000000000002499 g     F .text  000000000000001a              get_skbuff
0000000000015140 g       .bss 0000000000000000              _end
0000000000001110 g     F .text  000000000000002b              _start
00000000000024cd g     F .text  0000000000000024              read64
00000000000013b5 g     F .text  0000000000000078              bpf_lookup_elem
0000000000005090 g       .bss 0000000000000000              __bss_start
0000000000002779 g     F .text  0000000000000086              main
0000000000000000       F *UND*  0000000000000000              perror@@GLIBC_2.2.5
0000000000000000       F *UND*  0000000000000000              vfprintf@@GLIBC_2.2.5
0000000000001f99 g     F .text  00000000000000f4              redact
00000000000024b3 g     F .text  000000000000001a              get_fp
0000000000000000       F *UND*  0000000000000000              exit@@GLIBC_2.2.5
0000000000000000       F *UND*  0000000000000000              fwrite@@GLIBC_2.2.5
0000000000005090 g     O .data  0000000000000000              .hidden __TMC_END__
0000000000000000  w      *UND*  0000000000000000              _ITM_registerTMCloneTable
00000000000050cc g     O .bss 0000000000000004              doredact
0000000000000000       F *UND*  0000000000000000              strerror@@GLIBC_2.2.5
0000000000000000       F *UND*  0000000000000000              execl@@GLIBC_2.2.5
00000000000024f1 g     F .text  0000000000000028              write64
0000000000000000  w    F *UND*  0000000000000000              __cxa_finalize@@GLIBC_2.2.5
0000000000001e07 g     F .text  00000000000000c9              info
0000000000001ed0 g     F .text  00000000000000c9              msg
00000000000050c0 g     O .bss 0000000000000008              stderr@@GLIBC_2.2.5

If you’ve never looked at this output before, the column on the left is the symbol value and the far right column is the symbol name. The problem here is that none of these have 404 in them… anywhere. After much wasted time, I finally convinced myself this wasn’t the right binary, even though it looked shady as all get out and I’d be willing to bet it shows up again later.

Attempt to brute force the answer

You should know by this point that my answer to being in the wrong place would be to brute force the answer. I knew I was looking for an ELF binary, I suspected it should be somewhere near this Java download or the Java installation in usr/local/jdk1.8.0_151 where it ended up, and that it should have symbols which had 404 in it. I strongly suspect the 404 would be in the value, not the name, because of the hint that “leading 0’s don’t count”.

My approach was to chain together a few commands you should be used to and a few new ones. I had all three disk images mounted in /mnt so I went to that directory and ran a find command to list all the files benath it1. Because I wanted to limit what I was looking for, I used grep to limit my search to usr/local. I then piped the output to file and specified that it should read from standard input (-f -). From that, another grep would give us only ELF binaries and then we get to learn a new command.

The output of file has the filename at the start of the line, so I used awk to just print the first field ('{print $1}'), using “:” as the delimiter (-d ':'). This isn’t the best use of awk, cut probably would handle this just fine, but if you don’t know awk, this is a great place to learn as it is very powerful. For example, in other iterations I had awk displaying both the value and the name with '{print $1, $6}'. We then use a while loop to loop over each file, spit out the name, and then use objdump to flag anything with 000404 in it. The result is a nice long output that lets you quickly see which files have potential values and which words are shared by three of them in the same file.

[notta@cuppa /mnt]$ find \
| grep -v "src/" \
| grep "usr/local" \
| file -f - \
| grep ELF \
| awk -F ':' '{print $1}' \
| while read filename; do \
echo $filename && objdump -T $filename \
| grep 404; done

...[snip]...

./case2_slave1/usr/local/jdk1.8.0_151/jre/lib/amd64/libavplugin-53.so
./case2_master/usr/local/hadoop/bin/test-container-executor
00000000004048e0 g    DF .text  0000000000000053  Base        test_get_app_log_dir
0000000000404ab0 g    DF .text  0000000000000061  Base        test_check_configuration_permissions
0000000000404b20 g    DF .text  00000000000003aa  Base        test_delete_container
0000000000404ed0 g    DF .text  00000000000002ac  Base        test_delete_app
0000000000404800 g    DF .text  000000000000005d  Base        test_get_container_directory
00000000004044c0 g    DF .text  00000000000000d9  Base        create_nm_roots
00000000004045a0 g    DF .text  000000000000019e  Base        check_pid_file
0000000000404940 g    DF .text  00000000000000e4  Base        test_check_user
0000000000404740 g    DF .text  0000000000000053  Base        test_get_user_directory
00000000004047a0 g    DF .text  0000000000000058  Base        test_get_app_directory
0000000000404a30 g    DF .text  000000000000007a  Base        test_resolve_config_path
0000000000404860 g    DF .text  0000000000000072  Base        test_get_container_launcher_file
00000000004043c0 g    DF .text  00000000000000fb  Base        write_config_file
./case2_master/usr/local/hadoop/bin/container-executor
0000000000404a30 g    DF .text  000000000000001a  Base        get_values
0000000000404290 g    DF .text  000000000000007c  Base        free_configurations
00000000004044b0 g    DF .text  0000000000000432  Base        read_config
0000000000404aa0 g    DF .text  0000000000000067  Base        get_kv_key
0000000000404b10 g    DF .text  0000000000000087  Base        get_kv_value
0000000000404a50 g    DF .text  0000000000000015  Base        get_values_delim
0000000000404310 g    DF .text  0000000000000089  Base        resolve_config_path
00000000004048f0 g    DF .text  0000000000000070  Base        get_value
00000000004043a0 g    DF .text  000000000000010d  Base        check_configuration_permissions
0000000000404a70 g    DF .text  000000000000000a  Base        extract_values
0000000000404a80 g    DF .text  000000000000001a  Base        free_values
0000000000404960 g    DF .text  00000000000000c5  Base        extract_values_delim
./case2_master/usr/local/hadoop/lib/native/libhdfs.so.0.0.0
./case2_master/usr/local/hadoop/lib/native/libhadoop.so.1.0.0

...[snip]...

I found the same files I had found earlier and couldn’t find an appropriate word to submit (despite trying many).

Find the right place

Since we’ve exhausted brute force, I re-read the question for the 98th time. I was sure that the reference to binaries nearby was a reference to a bin folder at that point, so I looked in the java installation and checked for large fiiles (“elephants”). The largest binary was easy to find using ls with the sort argument (-S): /usr/local/jdk1.8.0_151/bin/unpack200. Specifically looking at that file showed three symbols which had 404 in the value, all sharing one word.

[notta@cuppa /mnt]$ ls -alS ./case2_master/usr/local/jdk1.8.0_151/bin

total 784
-rwxr-xr-x 1 snarky snarky 231482 Sep  5  2017 unpack200
-rwxr-xr-x 1 snarky snarky 129139 Sep  5  2017 javaws
-rwxr-xr-x 1 snarky snarky  74675 Nov 25  2016 jmc
-rwxr-xr-x 1 snarky snarky   8181 Sep  5  2017 tnameserv
-rwxr-xr-x 1 snarky snarky   8149 Sep  5  2017 orbd
-rwxr-xr-x 1 snarky snarky   8109 Sep  5  2017 jinfo
-rwxr-xr-x 1 snarky snarky   8109 Sep  5  2017 jmap
-rwxr-xr-x 1 snarky snarky   8109 Sep  5  2017 jstack
-rwxr-xr-x 1 snarky snarky   8013 Sep  5  2017 jconsole
-rwxr-xr-x 1 snarky snarky   7997 Sep  5  2017 policytool
...[snip]...

[notta@cuppa /mnt]$ objdump -t ./case2_master/usr/local/jdk1.8.0_151/bin/unpack200 | grep 404
0000000000404260 l     F .text  000000000000044a              deflate_fast
00000000004046b0 l     F .text  0000000000000557              deflate_stored
0000000000404c10 g     F .text  000000000000117d              deflate

The common word in all three of those is “deflate” and when you submit that as the answer, it works!

Find my problem

You’ll notice in my brute force above, this file did not show up, nor did these symbols. Why? Because I typo’d -T instead of -t. This came about from jumping between objdump and another tool (readelf) in which you need to use -T to view the entire symbol name. sigh Brute force didn’t let me down, I let brute force down.

Alternatives

Working brute force

If you fix the typo in my brute force, and add in use of sed to split apart the symbol names and uniq with the count (-c) option, you can have the same wall of text but have it tell you which words appear exactly 3 times in the fuile. This would have worked nicely, had I paid attention to my arguments.

[notta@cuppa case2_master]$ sudo find \
| grep "usr/local" \
| file -f - \
| grep ELF \
| awk -F ':' '{print $1}' \
| while read filename; do \
echo $filename && objdump -t $filename \
| grep 000404 \
| awk '{print $6}' \
| sed "s/_/\n/g" \
| sort \
| uniq -c | grep '3 '; done

./usr/local/hadoop/bin/test-container-executor
      3 app
      3 check
      3 container
      3 delete
      3 directory
      3 file
./usr/local/hadoop/bin/container-executor
./usr/local/hadoop/lib/native/libhdfs.so.0.0.0
./usr/local/hadoop/lib/native/libhadoop.so.1.0.0
./usr/local/hadoop/lib/native/examples/wordcount-nopipe
./usr/local/hadoop/lib/native/examples/wordcount-part
./usr/local/hadoop/lib/native/examples/wordcount-simple
./usr/local/hadoop/lib/native/examples/pipes-sort
./usr/local/jdk1.8.0_151/bin/jdb
./usr/local/jdk1.8.0_151/bin/javadoc
./usr/local/jdk1.8.0_151/bin/jconsole
./usr/local/jdk1.8.0_151/bin/jarsigner
./usr/local/jdk1.8.0_151/bin/jinfo
./usr/local/jdk1.8.0_151/bin/schemagen
./usr/local/jdk1.8.0_151/bin/jjs
./usr/local/jdk1.8.0_151/bin/java
./usr/local/jdk1.8.0_151/bin/appletviewer
./usr/local/jdk1.8.0_151/bin/rmic
./usr/local/jdk1.8.0_151/bin/tnameserv
./usr/local/jdk1.8.0_151/bin/jhat
./usr/local/jdk1.8.0_151/bin/wsimport
./usr/local/jdk1.8.0_151/bin/jmc
./usr/local/jdk1.8.0_151/bin/jrunscript
./usr/local/jdk1.8.0_151/bin/idlj
./usr/local/jdk1.8.0_151/bin/jps
./usr/local/jdk1.8.0_151/bin/policytool
./usr/local/jdk1.8.0_151/bin/javaws
./usr/local/jdk1.8.0_151/bin/javac
./usr/local/jdk1.8.0_151/bin/rmiregistry
./usr/local/jdk1.8.0_151/bin/jstat
./usr/local/jdk1.8.0_151/bin/native2ascii
./usr/local/jdk1.8.0_151/bin/orbd
./usr/local/jdk1.8.0_151/bin/javah
./usr/local/jdk1.8.0_151/bin/keytool
./usr/local/jdk1.8.0_151/bin/jmap
./usr/local/jdk1.8.0_151/bin/jstatd
./usr/local/jdk1.8.0_151/bin/pack200
./usr/local/jdk1.8.0_151/bin/jar
./usr/local/jdk1.8.0_151/bin/servertool
./usr/local/jdk1.8.0_151/bin/unpack200
      3 deflate
...[snip]...

In the output, you have a file I spent a lot of time on (test-container-executor) and the right file. There are not that many words to try, so this in theory should have worked nicely.

Following the directions

Assuming you were able to follow the question better than I did, you could also run a more targeted version that relies on ln to identify the largest file and head to only select that one file. This would be my preferred “solution in a Tweet” answer.

[notta@cuppa bin]$ filename=$(ls -S \
| head -n 1) && objdump -t $filename \
| grep 404 \
| awk '{print $6'} \
| sed "s/_/\n/g" \
| sort \
| uniq -c | grep '3 '

      3 deflate

Conclusion

This week was a lot more comfortable for me and fairly uplifting, compared to last week’s struggle with Hadoop. The command line is still a great place to be to solve these questions, all the more so if you actually provide the right input to it. If you’ve never used awk before, I’d definitely recommend you play with it this coming week.

Footnotes

  1. Because I had an eye towards making this fit into a Tweet later, I used find ... | grep instead of find -ipath... as it ends up being slightly shorter. ↩