linebreak.c File Reference

Implementation of the line breaking algorithm as described in Unicode Standard Annex 14. More...

#include <assert.h>
#include <stddef.h>
#include <string.h>
#include "linebreak.h"
#include "linebreakdef.h"
Include dependency graph for linebreak.c:

Data Structures

struct  LineBreakPropertiesIndex
 Struct for the second-level index to the line breaking properties. More...

Defines

#define LINEBREAK_INDEX_SIZE   40
 Size of the second-level index to the line breaking properties.

Enumerations

enum  BreakAction {
  DIR_BRK, IND_BRK, CMI_BRK, CMP_BRK,
  PRH_BRK
}
 

Enumeration of break actions.

More...

Functions

void init_linebreak (void)
 Initializes the second-level index to the line breaking properties.
static struct LineBreakPropertiesget_lb_prop_lang (const char *lang)
 Gets the language-specific line breaking properties.
static enum LineBreakClass get_char_lb_class (utf32_t ch, struct LineBreakProperties *lbp)
 Gets the line breaking class of a character from a line breaking properties array.
static enum LineBreakClass get_char_lb_class_default (utf32_t ch)
 Gets the line breaking class of a character from the default line breaking properties array.
static enum LineBreakClass get_char_lb_class_lang (utf32_t ch, struct LineBreakProperties *lbpLang)
 Gets the line breaking class of a character for a specific language.
static enum LineBreakClass resolve_lb_class (enum LineBreakClass lbc, const char *lang)
 Resolves the line breaking class for certain ambiguous or complicated characters.
utf32_t lb_get_next_char_utf8 (const utf8_t *s, size_t len, size_t *ip)
 Gets the next Unicode character in a UTF-8 sequence.
utf32_t lb_get_next_char_utf16 (const utf16_t *s, size_t len, size_t *ip)
 Gets the next Unicode character in a UTF-16 sequence.
utf32_t lb_get_next_char_utf32 (const utf32_t *s, size_t len, size_t *ip)
 Gets the next Unicode character in a UTF-32 sequence.
void set_linebreaks (const void *s, size_t len, const char *lang, char *brks, get_next_char_t get_next_char)
 Sets the line breaking information for a generic input string.
void set_linebreaks_utf8 (const utf8_t *s, size_t len, const char *lang, char *brks)
 Sets the line breaking information for a UTF-8 input string.
void set_linebreaks_utf16 (const utf16_t *s, size_t len, const char *lang, char *brks)
 Sets the line breaking information for a UTF-16 input string.
void set_linebreaks_utf32 (const utf32_t *s, size_t len, const char *lang, char *brks)
 Sets the line breaking information for a UTF-32 input string.
int is_line_breakable (utf32_t char1, utf32_t char2, const char *lang)
 Tells whether a line break can occur between two Unicode characters.

Variables

const int linebreak_version = LINEBREAK_VERSION
 Version number of the library.
static enum BreakAction baTable [LBP_JT][LBP_JT]
 Break action pair table.
static struct
LineBreakPropertiesIndex 
lb_prop_index [LINEBREAK_INDEX_SIZE]
 Second-level index to the line breaking properties.

Detailed Description

Implementation of the line breaking algorithm as described in Unicode Standard Annex 14.

Version:
2.1, 2011/05/07
Author:
Wu Yongwei

Define Documentation

#define LINEBREAK_INDEX_SIZE   40

Size of the second-level index to the line breaking properties.


Enumeration Type Documentation

Enumeration of break actions.

They are used in the break action pair table below.

Enumerator:
DIR_BRK 

Direct break opportunity.

IND_BRK 

Indirect break opportunity.

CMI_BRK 

Indirect break opportunity for combining marks.

CMP_BRK 

Prohibited break for combining marks.

PRH_BRK 

Prohibited break.


Function Documentation

static enum LineBreakClass get_char_lb_class ( utf32_t  ch,
struct LineBreakProperties lbp 
) [static]

Gets the line breaking class of a character from a line breaking properties array.

Parameters:
ch character to check
lbp pointer to the line breaking properties array
Returns:
the line breaking class if found; LBP_XX otherwise
static enum LineBreakClass get_char_lb_class_default ( utf32_t  ch  )  [static]

Gets the line breaking class of a character from the default line breaking properties array.

Parameters:
ch character to check
Returns:
the line breaking class if found; LBP_XX otherwise
static enum LineBreakClass get_char_lb_class_lang ( utf32_t  ch,
struct LineBreakProperties lbpLang 
) [static]

Gets the line breaking class of a character for a specific language.

This function will check the language-specific data first, and then the default data if there is no language-specific property available for the character.

Parameters:
ch character to check
lbpLang pointer to the language-specific line breaking properties array
Returns:
the line breaking class if found; LBP_XX otherwise
static struct LineBreakProperties* get_lb_prop_lang ( const char *  lang  )  [static, read]

Gets the language-specific line breaking properties.

Parameters:
lang language of the text
Returns:
pointer to the language-specific line breaking properties array if found; NULL otherwise
void init_linebreak ( void   ) 

Initializes the second-level index to the line breaking properties.

If it is not called, the performance of get_char_lb_class_lang (and thus the main functionality) can be pretty bad, especially for big code points like those of Chinese.

int is_line_breakable ( utf32_t  char1,
utf32_t  char2,
const char *  lang 
)

Tells whether a line break can occur between two Unicode characters.

This is a wrapper function to expose a simple interface. Generally speaking, it is better to use set_linebreaks_utf32 instead, since complicated cases involving combining marks, spaces, etc. cannot be correctly processed.

Parameters:
char1 the first Unicode character
char2 the second Unicode character
lang language of the input
Returns:
one of LINEBREAK_MUSTBREAK, LINEBREAK_ALLOWBREAK, LINEBREAK_NOBREAK, or LINEBREAK_INSIDEACHAR
utf32_t lb_get_next_char_utf16 ( const utf16_t s,
size_t  len,
size_t *  ip 
)

Gets the next Unicode character in a UTF-16 sequence.

The index will be advanced to the next complete character, unless the end of string is reached in the middle of a UTF-16 surrogate pair.

Parameters:
[in] s input UTF-16 string
[in] len length of the string in words
[in,out] ip pointer to the index
Returns:
the Unicode character beginning at the index; or EOS if end of input is encountered
utf32_t lb_get_next_char_utf32 ( const utf32_t s,
size_t  len,
size_t *  ip 
)

Gets the next Unicode character in a UTF-32 sequence.

The index will be advanced to the next character.

Parameters:
[in] s input UTF-32 string
[in] len length of the string in dwords
[in,out] ip pointer to the index
Returns:
the Unicode character beginning at the index; or EOS if end of input is encountered
utf32_t lb_get_next_char_utf8 ( const utf8_t s,
size_t  len,
size_t *  ip 
)

Gets the next Unicode character in a UTF-8 sequence.

The index will be advanced to the next complete character, unless the end of string is reached in the middle of a UTF-8 sequence.

Parameters:
[in] s input UTF-8 string
[in] len length of the string in bytes
[in,out] ip pointer to the index
Returns:
the Unicode character beginning at the index; or EOS if end of input is encountered
static enum LineBreakClass resolve_lb_class ( enum LineBreakClass  lbc,
const char *  lang 
) [static]

Resolves the line breaking class for certain ambiguous or complicated characters.

They are treated in a simplistic way in this implementation.

Parameters:
lbc line breaking class to resolve
lang language of the text
Returns:
the resolved line breaking class
void set_linebreaks ( const void *  s,
size_t  len,
const char *  lang,
char *  brks,
get_next_char_t  get_next_char 
)

Sets the line breaking information for a generic input string.

Parameters:
[in] s input string
[in] len length of the input
[in] lang language of the input
[out] brks pointer to the output breaking data, containing LINEBREAK_MUSTBREAK, LINEBREAK_ALLOWBREAK, LINEBREAK_NOBREAK, or LINEBREAK_INSIDEACHAR
[in] get_next_char function to get the next UTF-32 character
void set_linebreaks_utf16 ( const utf16_t s,
size_t  len,
const char *  lang,
char *  brks 
)

Sets the line breaking information for a UTF-16 input string.

Parameters:
[in] s input UTF-16 string
[in] len length of the input
[in] lang language of the input
[out] brks pointer to the output breaking data, containing LINEBREAK_MUSTBREAK, LINEBREAK_ALLOWBREAK, LINEBREAK_NOBREAK, or LINEBREAK_INSIDEACHAR
void set_linebreaks_utf32 ( const utf32_t s,
size_t  len,
const char *  lang,
char *  brks 
)

Sets the line breaking information for a UTF-32 input string.

Parameters:
[in] s input UTF-32 string
[in] len length of the input
[in] lang language of the input
[out] brks pointer to the output breaking data, containing LINEBREAK_MUSTBREAK, LINEBREAK_ALLOWBREAK, LINEBREAK_NOBREAK, or LINEBREAK_INSIDEACHAR
void set_linebreaks_utf8 ( const utf8_t s,
size_t  len,
const char *  lang,
char *  brks 
)

Sets the line breaking information for a UTF-8 input string.

Parameters:
[in] s input UTF-8 string
[in] len length of the input
[in] lang language of the input
[out] brks pointer to the output breaking data, containing LINEBREAK_MUSTBREAK, LINEBREAK_ALLOWBREAK, LINEBREAK_NOBREAK, or LINEBREAK_INSIDEACHAR

Variable Documentation

enum BreakAction baTable[LBP_JT][LBP_JT] [static]

Break action pair table.

This is a direct mapping of Table 2 of Unicode Standard Annex 14, Revision 24.

struct LineBreakPropertiesIndex lb_prop_index[LINEBREAK_INDEX_SIZE] [static]
Initial value:
{
    { 0xFFFFFFFF, lb_prop_default }
}

Second-level index to the line breaking properties.

const int linebreak_version = LINEBREAK_VERSION

Version number of the library.


Generated by  doxygen 1.6.2