JSON response full of dots “.” after ‘Follow TCP Stream’

Question

I'm trying to follow a JSON response but it shows it offuscated, this is the full Raw TCP Stream Wireshark is showing me:

GET /api/v1/dependencies?gems=boolean_class,bundler,rake HTTP/1.1
Accept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3
Accept: */*
User-Agent: bundler/1.5.0.rc.1 rubygems/2.1.11 ruby/2.0.0 (x86_64-unknown-linux-gnu) command/install 273d73b77583ab06
Connection: keep-alive
Keep-Alive: 30
Host: 127.0.1.1:3132
HTTP/1.1 200 OK
Content-Type: text/html;charset=utf-8
X-Powered-By: geminabox 0.12.1
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
Content-Length: 750
..[.{.:.nameI".boolean_class.:.ET:.numberI"
0.0.3.;.T:
platformI".ruby.;.F:.dependencies[.{.;.I".rake.;.T;.I".10.1.0.;.T;.I".ruby.;.F;.[.{.;.I"
rspec.;.T;.I".2.14.1.;.T;.I".ruby.;.F;.[.[.I".rspec-core.;.TI".~> 2.14.0.;.F[.I".rspec-expectations.;.TI".~> 2.14.0.;.F[.I".rspec-mocks.;………………….. etc ………

I tried ASCII what got the same.

Accepted Answer

1

The dots are "non-printable" characters (i.e. there is no ASCII representation of them). They are likely to be binary data values. The hex display pane will show the the exact values.

answered 01 Dec '13, 08:44

grahamb ♦
19.8k●3●30●206
accept rate: 22%

You are right @grahamb !! i wrongly assumed it was text.

If i wget it then open the binary file using Sublime Text with utf-8 encoding somehow it figures what the symbols mean:

alt text

Perhaps as a nice to have feature Wireshark may show this utf-8 symbols instead of converting them to dots "."

(01 Dec '13, 09:23) elgalu

You are right!! Perhaps as a nice to have feature Wireshark may show this utf-8 symbols instead of converting them to dots "." See Sublime Text screenshot below. Thanks!

(01 Dec '13, 09:24) elgalu

1

Actually, the things such as EOT, BS, ACK & etc are just the ASCII names for certain ASCII non-printing ("control") characters. Some of the names (e.g., ACK, DC1, DC2) are basically historical and others are things like (BS [backspace], VT [vertical tab]); In any case the names probably don't reflect the actual meaning/usage of the underlying binary value.

IOW: this has nothing to do with UTF-8.

(01 Dec '13, 09:36) Bill Meier ♦♦

Thanks for all the clarifications! Very helpful.

(01 Dec '13, 09:39) elgalu

The reason why there are non-printable characters is: that's not JSON. It looks like BSON (binary JSON), which contains 'binary characters/values' as the name already implies.

Regards
Kurt

(01 Dec '13, 10:17) Kurt Knochner ♦

After digging into the source code i realized it was Ruby binary serialized data through Marshal easily deserialized through Marshal.load(File.open('not-json.binary'))

Marshal.load(File.open('not-json.binary')) #=>
[ {:name=>"boolean_class", :number=>"0.0.3", :platform=>"ruby", :dependencies=>[]},
{:name=>"rake", :number=>"10.1.0", :platform=>"ruby", :dependencies=>[]},
{:name=>"rspec", :number=>"2.14.1", :platform=>"ruby", :dependencies=> [
["rspec-core", "~> 2.14.0"], ["rspec-expectations", "~> 2.14.0"], ["rspec-mocks", "~> 2.14.0"]
]}
]

(03 Dec ‘13, 10:18) elgalu

showing 5 of 6 show 1 more comments