Why does Java’s hashCode() in String use 31 as a multiplier?

The value 31 was chosen because it is an odd prime. If it were even and the multiplication overflowed, information would be lost, as multiplication by 2 is equivalent to shifting. The advantage of using a prime is less clear, but it is traditional. A nice property of 31 is that the multiplication can be replaced by a shift and a subtraction for better performance: 31 * i == (i << 5) - i. Modern VMs do this sort of optimization automatically.

(from Chapter 3, Item 9: Always override hashcode when you override equals, page 48, Joshua Bloch’s Effective Java)

27 Temmuz 2016

Posted In: effectivejava, hashcode, java, string

How to build Hadoop Native Library with Snappy Compression Support

Snappy is a compression library that can be utilized by the native code.
It is currently an optional component, meaning that Hadoop can be built with
or without this dependency.

Download and compile snappy codecs. or you can install from your distro repo. I installed libsnappy and libsnappy-dev packages from Ubuntu repo. If everything is fine you can use -Drequire.snappy to fail the build if libsnappy.so is not found. If this option is not specified and the snappy library is missing,silently build a version of libhadoop.so that cannot make use of snappy. After than You just need to enter below command:

mvn clean package -Pdist,native -DskipTests -Dtar -Drequire.snappy

If you build snappy and It is located different place you can use this parameters

  • -Dsnappy.prefix to specify a nonstandard location for the libsnappy header files and library files. You do not need this option if you have installed snappy using a package manager.
  • -Dsnappy.lib to specify a nonstandard location for the libsnappy library files. Similarly to snappy.prefix, you do not need this option if you have installed snappy using a package manager.
  • -Dbundle.snappy to copy the contents of the snappy.lib directory into the final tar file. This option requires that -Dsnappy.lib is also given, and it ignores the -Dsnappy.prefix option.

After compiling finished you can find your native libraries 

<source_folder>/hadoop-dist/target/hadoop-2.5.2/lib/native/

Good luck

13 Temmuz 2016

Posted In: hadoop, hadoop-native, snappy

Twitter Auto Publish Powered By : XYZScripts.com