2021년 2월 17일 (수) 08:28에 Pythagoras0님의 편집

2021-02-17T08:28:46Z

Pythagoras0: /* 메타데이터 */ 새 문단

2020-12-26T12:27:33Z

메타데이터: 새 문단

Pythagoras0: /* 노트 */ 새 문단

2020-12-21T05:01:17Z

노트: 새 문단

새 문서

== 노트 ==

===위키데이터===
* ID : [https://www.wikidata.org/wiki/Q193537 Q193537]
===말뭉치===
# The first 128 UTF-8 characters precisely match the first 128 ASCII characters (numbered 0-127), meaning that existing ASCII text is already valid UTF-8.<ref name="ref_31c05569">[https://developer.mozilla.org/en-US/docs/Glossary/UTF-8 UTF-8 - MDN Web Docs Glossary: Definitions of Web-related terms]</ref>
# A character in UTF8 can be from 1 to 4 bytes long.<ref name="ref_53572a62">[https://www.w3schools.com/charsets/ref_html_utf8.asp HTML UTF-8 Reference]</ref>
# UTF-8 can represent any character in the Unicode standard.<ref name="ref_53572a62" />
# HTML 4 supports UTF-8.<ref name="ref_53572a62" />
# One of the really nice features of UTF-8 is that it is compatible with nul-terminated strings.<ref name="ref_18dd92dc">[https://www.fileformat.info/info/unicode/utf8.htm UTF-8 Encoding]</ref>
# For characters equal to or below 2047 (hex 0x07FF), the UTF-8 representation is spread across two bytes.<ref name="ref_18dd92dc" />
# UTF-8 remains a simple, single-byte, ASCII-compatible encoding method, as long as no characters greater than 127 are directly present.<ref name="ref_18dd92dc" />
# A: Yes, there are several possible representations of Unicode data, including UTF-8, UTF-16 and UTF-32.<ref name="ref_396e3afb">[https://unicode.org/faq/utf_bom.html UTF-8, UTF-16, UTF-32 & BOM]</ref>
# For example, in UTF-8 every byte of the form 110xxxxx 2 must be followed with a byte of the form 10xxxxxx 2 .<ref name="ref_396e3afb" />
# Latin-1. UTF-8 uses the bytes in the ASCII only for ASCII characters.<ref name="ref_396e3afb" />
# UTF-8 is the byte-oriented encoding form of Unicode.<ref name="ref_396e3afb" />
# In this post, I’ll explain the basics of one technology central to text on the web, UTF-8.<ref name="ref_96563f48">[https://blog.hubspot.com/website/what-is-utf-8 What is UTF-8 Encoding? A Guide for Non-Programmers]</ref>
# UTF-8 is an encoding system for Unicode.<ref name="ref_96563f48" />
# There are other encoding systems for Unicode besides UTF-8, but UTF-8 is unique because it represents characters in one-byte units.<ref name="ref_96563f48" />
# More specifically, UTF-8 converts a code point (which represents a single character in Unicode) into a set of one to four bytes.<ref name="ref_96563f48" />
# The originally proposed encodings of the UCS, however, were not compatible with many current applications and protocols, and this has led to the development of UTF-8, the object of this memo.<ref name="ref_53db38df">[https://tools.ietf.org/html/rfc3629 UTF-8, a transformation format of ISO 10646]</ref>
# All standard UCS encoding forms except UTF-8 have an encoding unit larger than one octet, making them hard to use in many current applications and protocols that assume 8 or even 7 bit characters.<ref name="ref_53db38df" />
# CESU-8 operates similarly to UTF-8 but encodes the UTF-16 code values (16-bit quantities) instead of the character number (code point).<ref name="ref_53db38df" />
# This leads to different results for character numbers above 0xFFFF; the CESU-8 encoding of those characters is NOT valid UTF-8.<ref name="ref_53db38df" />
# UTF-8 is a variable-width character encoding used for electronic communication.<ref name="ref_0afa1de8">[https://en.wikipedia.org/wiki/UTF-8 Wikipedia]</ref>
# The Use of the main encodings on the web from 2001 to 2012 as recorded by Google,with UTF-8 overtaking all others in 2008 and over 60% of the web in 2012.<ref name="ref_0afa1de8" />
# The World Wide Web Consortium recommends UTF-8 as the default encoding in XML and HTML (and not just using UTF-8, also stating it in metadata), "even when all characters are in the ASCII range ..<ref name="ref_0afa1de8" />
# In locales where UTF-8 is used alongside another encoding, the latter is typically more efficient for the associated language.<ref name="ref_0afa1de8" />
# UTF-8 and Unicode Unicode Transformation Format 8-bit is a variable-width encoding that can represent every character in the Unicode character set.<ref name="ref_c659fafb">[http://www.utf-8.com/ UTF-8 and Unicode Standards]</ref>
# UTF-8 encodes each Unicode character as a variable number of 1 to 4 octets, where the number of octets depends on the integer value assigned to the Unicode character.<ref name="ref_c659fafb" />
# The MIME character set attribute for UTF-8 is UTF-8 .<ref name="ref_c659fafb" />
# UTF-8 is a variable-width encoding that can represent every character in the Unicode character set.<ref name="ref_451a2d11">[https://www.toptal.com/php/a-utf-8-primer-for-php-and-mysql A Guide to UTF-8 Encoding in PHP and MySQL]</ref>
# UTF-8 encodes each character using one to four bytes.<ref name="ref_451a2d11" />
# A is U+0041, which in UTF-8 is simply encoded with the single byte 41.<ref name="ref_451a2d11" />
# In comparison, the Unicode hexidecimal code for the character is U+233B4, which in UTF-8 is encoded with the four bytes F0 A3 8E B4.<ref name="ref_451a2d11" />
# , how UTF-8 decoders handle various types of | corrupted or otherwise interesting UTF-8 sequences.<ref name="ref_733f8cdf">[https://www.w3.org/2001/06/utf-8-wrong/UTF-8-test.html UTF-8 test file]</ref>
# According to ISO 10646-1, sections R.7 and 2.3c, a device receiving | UTF-8 shall interpret a "malformed sequence in the same way that it | interprets a character that is outside the adopted subset".<ref name="ref_733f8cdf" />
# This means | usually that the malformed UTF-8 sequence is replaced by a replacement | character (U+FFFD), which looks a bit like an inverted question mark, | or a similar symbol.<ref name="ref_733f8cdf" />
# It might be a good idea to visually distinguish a | malformed UTF-8 sequence from a correctly encoded Unicode character | that is just not available in the current font but otherwise fully | legal.<ref name="ref_733f8cdf" />
# The browser interprets those numbers as UTF-8, and internally converts them into Unicode code points.<ref name="ref_801158f2">[https://www.smashingmagazine.com/2012/06/all-about-unicode-utf8-character-sets/ Unicode, UTF8 & Character Sets: The Ultimate Guide — Smashing Magazine]</ref>
# If you display the page using the UTF-8 character set, you will see only 3 characters: HЯ⾀.<ref name="ref_801158f2" />
# UTF-8 is becoming the most popular international character set on the Internet, superseding the older single-byte character sets like ISO-8859-5.<ref name="ref_801158f2" />
# Perhaps the Ð looks familiar - it will sometimes show up if you try to view Russian UTF-8 documents.<ref name="ref_801158f2" />
# UTF-8 uses one byte to represent code points from 0-127.<ref name="ref_a3ebcda7">[https://www.twilio.com/docs/glossary/what-utf-8 What is UTF-8?]</ref>
# The first UTF-8 byte signals how many bytes will follow it.<ref name="ref_a3ebcda7" />
# UTF-8 pads the leading bits with three 0 s to fully “fill out” the remaining spaces.<ref name="ref_a3ebcda7" />
# The en_US.UTF-8 locale provides multiscript processing support by using UTF-8 as its codeset.<ref name="ref_ac5097ae">[https://docs.oracle.com/cd/E19683-01/806-6642/utf8-21349/index.html Chapter 5 Overview of UTF-8 Locale Support (International Language Environments Guide)]</ref>
# After you install the Japanese locale, you can use ATOK12 in all UTF-8 locales.<ref name="ref_ac5097ae" />
# If your message contains characters from a mixture of scripts, the default MIME charset is UTF-8 .<ref name="ref_ac5097ae" />
# Any 8-bit characters of UTF-8 are encoded with Quoted-Printable encoding.<ref name="ref_ac5097ae" />
# The "I can eat glass" phrase and initial translations (about 30 of them) were borrowed from Ethan Mollick's I Can Eat Glass page (which disappeared on or about June 2004) and converted to UTF-8.<ref name="ref_cdcdcf32">[http://www.columbia.edu/~fdc/utf8/ UTF-8 Sampler]</ref>
# Kermit 95 displays UTF-8 and also allows keyboard entry of arbitrary Unicode BMP characters as 4 hex digits, as shown HERE.<ref name="ref_cdcdcf32" />
# EMACS 21.1 actually supports UTF-8; earlier versions don't know about it and display the octal codes; either way is OK for this purpose.<ref name="ref_cdcdcf32" />
# UltraEdit / UEStudio provides support for Unicode (16-bit wide character, or UTF-16) and UTF-8 files.<ref name="ref_31bf2c6d">[https://www.ultraedit.com/wiki/Unicode_/_UTF-8_support Unicode / UTF-8 support]</ref>
# You can directly edit UTF-8 and UTF-16 files and convert them between ANSI and Unicode formats.<ref name="ref_31bf2c6d" />
# String occurrences "charset=utf-8" or "encoding=utf-8" in the file.<ref name="ref_31bf2c6d" />
# If the file is determined to be UTF-8, it will be treated as such and on open, it will be converted internally to Unicode (16-bit) for editing.<ref name="ref_31bf2c6d" />
# This report shows the usage statistics of UTF-8 as character encoding on the web.<ref name="ref_7bb7471d">[https://w3techs.com/technologies/details/en-utf8 Usage Statistics and Market Share of UTF-8 for Websites, December 2020]</ref>
# UTF-8 is a byte oriented encoding.<ref name="ref_9575819d">[https://docs.python.org/3/howto/unicode.html Unicode HOWTO — Python 3.9.1 documentation]</ref>
# UTF-8 is one of the most commonly used encodings, and Python often defaults to using it.<ref name="ref_9575819d" />
# If you don’t include such a comment, the default encoding used will be UTF-8 as already mentioned.<ref name="ref_9575819d" />
# Python supports writing source code in UTF-8 by default, but you can use almost any encoding if you declare the encoding being used.<ref name="ref_9575819d" />
# WebSEAL implements multi-locale support by internally maintaining and handling all data using UCS Transformation Format 8 byte (UTF-8) encoding .<ref name="ref_2942fe98">[https://publib.boulder.ibm.com/tividd/td/ITAME/SC32-1359-00/en_US/HTML/am51_webseal_guide24.htm Multi-locale support with UTF-8]</ref>
# WebSEAL handles data internally in UTF-8 regardless of the locale in which the WebSEAL process is running.<ref name="ref_2942fe98" />
# Note that most operating systems do not use UTF-8 by default.<ref name="ref_2942fe98" />
# Local code pages can by UTF-8 or not UTF-8.<ref name="ref_2942fe98" />
# The UTF-8 encoding of Unicode and UCS does not have these problems and is the common way in which Unicode is used on UNIX-style operating systems.<ref name="ref_8793ed80">[https://man7.org/linux/man-pages/man7/UTF-8.7.html Linux manual page]</ref>
# All possible 2^31 UCS codes can be encoded using UTF-8.<ref name="ref_8793ed80" />
# The bytes 0xc0, 0xc1, 0xfe, and 0xff are never used in the UTF-8 encoding.<ref name="ref_8793ed80" />
# * UTF-8 encoded UCS characters may be up to six bytes long, however the Unicode standard specifies no characters above 0x10ffff, so Unicode characters can be only up to four bytes long in UTF-8.<ref name="ref_8793ed80" />
# Rust’s propensity for exposing possible errors, strings being a more complicated data structure than many programmers give them credit for, and UTF-8.<ref name="ref_68c3396a">[https://doc.rust-lang.org/book/ch08-02-strings.html The Rust Programming Language]</ref>
# In Chapter 4, we talked about string slices, which are references to some UTF-8 encoded string data stored elsewhere.<ref name="ref_68c3396a" />
# a growable, mutable, owned, UTF-8 encoded string type.<ref name="ref_68c3396a" />
# Let’s look at some of our properly encoded UTF-8 example strings from Listing 8-14.<ref name="ref_68c3396a" />
# utf-8 , where utf is short for unicode transformation format, is a method of encoding unicode characters using one to four bytes per character.<ref name="ref_217efd5e">[https://wiki.tcl-lang.org/page/utf-8 utf-8]</ref>
# Internally, Tcl uses modified utf-8 encoding, which is the same as utf-8 except that the NUL character (\u0000) is encoded as the bytes 0xC0 0x80, which is not a valid utf-8 sequence.<ref name="ref_217efd5e" />
# This doesn't play well with encoding convertto utf-8 though, as that will reencode each surrogate in the pair as a separate character.<ref name="ref_217efd5e" />
# Overview ▾ Package utf8 implements functions and constants to support text encoded in UTF-8.<ref name="ref_89fd9dac">[https://golang.org/pkg/unicode/utf8/ The Go Programming Language]</ref>
# It includes functions to translate between runes and UTF-8 byte sequences.<ref name="ref_89fd9dac" />
# DecodeLastRune unpacks the last UTF-8 encoding in p and returns the rune and its width in bytes.<ref name="ref_89fd9dac" />
# An encoding is invalid if it is incorrect UTF-8, encodes a rune that is out of range, or is not the shortest possible UTF-8 encoding for the value.<ref name="ref_89fd9dac" />
# Create a UTF-8 encoding.<ref name="ref_b5bdb900">[https://docs.microsoft.com/en-us/dotnet/api/system.text.utf8encoding UTF8Encoding Class (System.Text)]</ref>
# Text Class Example Public Shared Sub Main() ' Create a UTF-8 encoding.<ref name="ref_b5bdb900" />
# Create a UTF-8 encoding that supports a BOM.<ref name="ref_b5bdb900" />
# Text Class Example Public Shared Sub Main() ' Create a UTF-8 encoding that supports a BOM.<ref name="ref_b5bdb900" />
# UTF-8 is a standard for representing Unicode numbers in computer files.<ref name="ref_244d18c8">[https://www.hesa.ac.uk/support/user-guides/xml-files/unicode Common Unicode and UTF-8 issues]</ref>
# Therefore it is good practice to prefix a UTF-8 file with three special bytes, called the Byte Order Mark header (BOM header).<ref name="ref_244d18c8" />
# The HESA data collection system always outputs its UTF-8 files with BOM headers.<ref name="ref_244d18c8" />
# It is strongly recommended that institutions use UTF-8 BOM headers in their submitted XML files.<ref name="ref_244d18c8" />
# So, the first UTF-8 byte is used for encoding ASCII, giving the character set full backwards compatibility with ASCII.<ref name="ref_6bed847b">[https://wiki.gentoo.org/wiki/UTF-8 Gentoo Wiki]</ref>
# UTF-8 means that ASCII and Latin characters are interchangeable with little increase in the size of the data, because only the first byte is used.<ref name="ref_6bed847b" />
# UTF-8 allows users to work in a standards-compliant and internationally accepted multilingual environment, with a comparatively low data redundancy.<ref name="ref_6bed847b" />
# Despite this, many people regard UTF-8 in online communication as abusive.<ref name="ref_6bed847b" />
# For more information, see Section 10.9.2, “The utf8mb3 Character Set (3-Byte UTF-8 Unicode Encoding)”.<ref name="ref_892f07eb">[https://dev.mysql.com/doc/refman/5.6/en/charset-unicode-utf8.html MySQL :: MySQL 5.6 Reference Manual :: 10.9.3 The utf8 Character Set (Alias for utf8mb3)]</ref>
# Converts a string encoded in ANSI to UTF-8 with a given code page.<ref name="ref_87d6dd89">[http://docwiki.embarcadero.com/RADStudio/Sydney/en/UTF-8_Conversion_Routines UTF-8 Conversion Routines]</ref>
# One benefit of UTF-8 is its ability to deal with languages that have 100s and 1000s of characters.<ref name="ref_d4f04d65">[https://docs.moodle.org/en/UTF-8 MoodleDocs]</ref>
# UTF-8 is a clever way of encoding Unicode text.<ref name="ref_c5571263">[https://www.johndcook.com/blog/2019/09/09/how-utf-8-works/ How UTF-8 Unicode encoding works]</ref>
# I’ve mentioned it a couple times lately, but I haven’t blogged about UTF-8 per se.<ref name="ref_c5571263" />
# UTF-8 is a way of encoding Unicode so that an ASCII text file encodes to itself.<ref name="ref_c5571263" />
# When software reading UTF-8 comes across a byte starting with 1, it counts how many 1’s follow before encountering a 0.<ref name="ref_c5571263" />
# The use utf8 pragma tells the Perl parser to allow UTF-8 in the program text in the current lexical scope.<ref name="ref_0fa95d6e">[https://perldoc.perl.org/utf8 Perl pragma to enable/disable UTF-8 (or UTF-EBCDIC) in source code]</ref>
# The no utf8 pragma tells Perl to switch back to treating the source text as literal bytes in the current lexical scope.<ref name="ref_0fa95d6e" />
# Do not use this pragma for anything else than telling Perl that your script is written in UTF-8.<ref name="ref_0fa95d6e" />
# Bytes in the source text that are not in the ASCII character set will be treated as being part of a literal UTF-8 sequence.<ref name="ref_0fa95d6e" />
# UTF-8 uses an 8-bit code unit, and UTF-16 uses a 16-bit code unit.<ref name="ref_64c426a7">[https://flaviocopes.com/unicode/ Introduction to Unicode and UTF-8]</ref>
# UTF-8 was designed to be backward compatible with ASCII.<ref name="ref_64c426a7" />
# UTF-16 is a variable length encoding system, like UTF-8, but uses 2 bytes (16 bits) as the minimum for any character representation.<ref name="ref_64c426a7" />
# However, early version of the UTF-8 specification got some entries wrong (in some cases it permitted overlong characters).<ref name="ref_f60445ec">[https://capec.mitre.org/data/definitions/80.html CAPEC-80: Using UTF-8 Encoding to Bypass Validation Logic (Version 3.3)]</ref>
# UTF-8 encoders are supposed to use the "shortest possible" encoding, but naive decoders may accept encodings that are longer than necessary.<ref name="ref_f60445ec" />
# Techniques Try to use UTF-8 encoding of content in Scripts in order to bypass validation routines.<ref name="ref_f60445ec" />
# Try to use UTF-8 encoding of content in HTML in order to bypass validation routines.<ref name="ref_f60445ec" />
# Now known as “UTF-8”, FSS-UTF was essentially completed.<ref name="ref_bf56866f">[https://www.ionos.com/digitalguide/websites/website-creation/utf-8-encoding-global-digital-communication/ UTF-8: the network standard]</ref>
# The first argument is a character encoding name, like "UTF-8" or "ASCII" or "EUC-JP".<ref name="ref_8d7a4963">[https://stackoverflow.com/questions/6162484/why-does-modern-perl-avoid-utf-8-by-default Why does modern Perl avoid UTF-8 by default?]</ref>
# Database drivers can be flaky; if you use DBD::SQLite with just Perl, it will work out, but if some other tool has put text stored as some encoding other than UTF-8 in your database...<ref name="ref_8d7a4963" />
# Unless you say use utf8 at the top of each file, Perl will not assume that your source code is UTF-8.<ref name="ref_8d7a4963" />
# It prints the UTF-8 data with a poo at the end of each line.<ref name="ref_8d7a4963" />
===소스===
<references />

@@ 118번째 줄: / 118번째 줄: @@
   <references />
-== 메타데이터 ==
+==메타데이터==
 ===위키데이터===
 * ID :  [https://www.wikidata.org/wiki/Q193537 Q193537]

← 이전 판		2020년 12월 26일 (토) 12:27 판
117번째 줄:		117번째 줄:
	===소스===		===소스===
	<references />		<references />
		+
		+	== 메타데이터 ==
		+
		+	===위키데이터===
		+	* ID : [https://www.wikidata.org/wiki/Q193537 Q193537]

UTF-8 - 편집 역사

2021년 2월 17일 (수) 08:28에 Pythagoras0님의 편집

Pythagoras0: /* 메타데이터 */ 새 문단

Pythagoras0: /* 노트 */ 새 문단