# libmojibake

[libmojibake](https://github.com/JuliaLang/libmojibake) is
a lightly updated fork of the [utf8proc
library](http://www.public-software-group.org/utf8proc) from Jan
Behrens and the rest of the [Public Software
Group](http://www.public-software-group.org/), who deserve *nearly all
of the credit* for this package: a small, clean C library that
provides Unicode normalization, case-folding, and other operations for
data in the [UTF-8 encoding](http://en.wikipedia.org/wiki/UTF-8).

The reason for this fork is that `utf8proc` is used for basic Unicode
support in the [Julia language](http://julialang.org/) and the Julia
developers wanted Unicode 7 support and other features, but the
Public Software Group currently does not seem to have the resources
necessary to update `utf8proc`.  We hope that the fork can be merged
back into the mainline `utf8proc` package before too long.

(The original `utf8proc` package also includes Ruby and PostgreSQL plug-ins.
We removed those from `libmojibake` in order to focus exclusively on the C
library for the time being.  We will strive to keep API changes to a minimum,
so `libmojibake` should still be usable with the old plug-in code.)

Like `utf8proc`, the `libmojibake` package is licensed under the
free/open-source [MIT "expat"
license](http://opensource.org/licenses/MIT) (plus certain Unicode
data governed by the similarly permissive [Unicode data
license](http://www.unicode.org/copyright.html#Exhibit1)); please see
the included `LICENSE.md` file for more detailed information.

## Quick Start ##

For compilation of the C library run `make`.

## General Information ##

The C library is found in this directory after successful compilation
and is named `libmojibake.a` (for the static library) and
`libmojibake.so` (for the dynamic library).

The Unicode version being supported is 5.0.0.
*Note:* Version 4.1.0 of Unicode Standard Annex #29 was used, as
version 5.0.0 had not been available at the time of implementation.

For Unicode normalizations, the following options are used:

* Normalization Form C:  `STABLE`, COMPOSE`
* Normalization Form D:  `STABLE`, `DECOMPOSE`
* Normalization Form KC: `STABLE`, `COMPOSE`, `COMPAT`
* Normalization Form KD: `STABLE`, `DECOMPOSE`, `COMPAT`

## C Library ##

The documentation for the C library is found in the `utf8proc.h` header file.
`utf8proc_map` is function you will most likely be using for mapping UTF-8
strings, unless you want to allocate memory yourself.

## To Do ##

* detect stable code points and process segments independently in order to save memory
* do a quick check before normalizing strings to optimize speed
* support stream processing

## Contact ##

Bug reports, feature requests, and other queries can be filed at
the [libmojibake page on Github](https://github.com/JuliaLang/libmojibake/issues).