Implement seek() utfbytes() in Sys module. - Updated tests for utfbytes() - Updated utf8 API surface to support the general length case (codepoint _and_ byte) - Testing of seek() still outstanding Comment added regarding stack frame pointer in stack frames.

arr committed 9 years ago

Commit a6da1dc1 parent 1af55286 files · +94 −18Browse files

Showing the changes to src/include (1 of 6 files). Show all

Unified · Split

src/include/utf8.h+9−2

@@ -29,9 +29,16 @@ namespace disvm · 28 unmodified lines
29	29	const disvm::runtime::rune_t max_supported_codepoint = max_codepoint_4_byte;
30	30	};
31	31
32		− // Compute the number of code points in the supplied string
	32	+ // Lengths of a UTF-8 string
	33	+ struct utf8_length_t
	34	+ {
	35	+ const std::size_t codepoint_count;
	36	+ const std::size_t byte_count;
	37	+ };
	38	+
	39	+ // Compute the number of codepoints in the supplied string
33	40	// See http://bjoern.hoehrmann.de/utf-8/decoder/dfa/ for details.
34		− std::size_t count_codepoints(const uint8_t *str, const std::size_t max_len);
	41	+ utf8_length_t count_codepoints(const uint8_t *str, const std::size_t max_len);
35	42
36	43	enum class decode_state_t
37	44	{