diff --git a/src/docs/user/userguide/remarkup.diviner b/src/docs/user/userguide/remarkup.diviner index 8b4c0e39b2..9f1e54e448 100644 --- a/src/docs/user/userguide/remarkup.diviner +++ b/src/docs/user/userguide/remarkup.diviner @@ -1,504 +1,505 @@ @title Remarkup Reference @group userguide Explains how to make bold text; this makes your words louder so you can win arguments. = Overview = Phabricator uses a lightweight markup language called "Remarkup", similar to other lightweight markup languages like Markdown and Wiki markup. This document describes how to format text using Remarkup. = Quick Reference = All the syntax is explained in more detail below, but this is a quick guide to formatting text in Remarkup. These are inline styles, and can be applied to most text: - **bold** //italic// ##monospaced## `monospaced` ~~deleted~~ + **bold** //italic// ##monospaced## `monospaced` ~~deleted~~ __underlined__ D123 T123 rX123 # Link to Objects {D123} {T123} # Link to Objects (Full Name) {F123} # Embed Images @username # Mention a user [[wiki page]] # Link to Phriction [[wiki page | name]] # Named link to Phriction http://xyz/ # Link to web [[http://xyz/ | name]] # Named link to web [name](http://xyz/) # Alternate Link These are block styles, and must be separated from surrounding text by empty lines: = Large Header = == Smaller Header == Also a Large Header =================== Also a Smaller Header --------------------- > Quoted Text Use "- " or "* " for bulleted lists, and "# " for numbered lists. Use ``` or indent two spaces for code. Use %%% for a literal block. Use | ... | ... for tables. = Basic Styling = Format **basic text styles** like this: **bold text** //italic text// ##monospaced text## `monospaced text` ~~deleted text~~ + __underlined text__ Those produce **bold text**, //italic text//, ##monospaced text##, -`monospaced text` and ~~deleted text~~, respectively. +`monospaced text`, ~~deleted text~~, and __underlined text__, respectively. = Layout = Make **headers** like this: = Large Header = == Smaller Header == ===== Very Small Header ===== Alternate Large Header ====================== Alternate Smaller Header ------------------------ You can optionally omit the trailing `=` signs -- that is, these are the same: == Smaller Header == == Smaller Header This produces headers like the ones in this document. Make sure you have an empty line before and after the header. Make **lists** by beginning each item with a "-" or a "*": lang=text - milk - eggs - bread * duck * duck * goose This produces a list like this: - milk - eggs - bread (Note that you need to put a space after the "-" or "*".) You can make numbered lists with a "#" instead of "-" or "*": # Articuno # Zapdos # Moltres You can also nest lists: ```- Body - Head - Arm - Elbow - Hand # Thumb # Index # Middle # Ring # Pinkie - Leg - Knee - Foot``` ...which produces: - Body - Head - Arm - Elbow - Hand # Thumb # Index # Middle # Ring # Pinkie - Leg - Knee - Foot If you prefer, you can indent lists using multiple characters to show indent depth, like this: ```- Tree -- Branch --- Twig``` As expected, this produces: - Tree -- Branch --- Twig Make **code blocks** by indenting two spaces: f(x, y); You can also use three backticks to enclose the code block: ```f(x, y); g(f);``` You can specify a language for syntax highlighting with "lang=xxx": lang=text lang=html ... This will highlight the block using a highlighter for that language, if one is available (in most cases, this means you need to configure Pygments): lang=html ... You can also use a "COUNTEREXAMPLE" header to show that a block of code is bad and shouldn't be copied: lang=text COUNTEREXAMPLE function f() { global $$variable_variable; } This produces a block like this: COUNTEREXAMPLE function f() { global $$variable_variable; } You can use ##lines=N## to limit the vertical size of a chunk of code, and ##name=some_name.ext## to give it a name. For example, this: lang=text lang=html, name=example.html, lines=12, counterexample ... ...produces this: lang=html, name=example.html, lines=12, counterexample

Apple

Apricot

Avocado

Banana

Bilberry

Blackberry

Blackcurrant

Blueberry

Currant

Cherry

Cherimoya

Clementine

Date

Damson

Durian

Eggplant

Elderberry

Feijoa

Gooseberry

Grape

Grapefruit

Guava

Huckleberry

Jackfruit

Jambul

Kiwi fruit

Kumquat

Legume

Lemon

Lime

Lychee

Mandarine

Mango

Mangostine

Melon

You can also use "NOTE:" to call out an important idea. NOTE: Don't cross the streams! = Linking URIs = URIs are automatically linked: http://phabricator.org/ If you have a URI with problematic characters in it, like "##http://comma.org/,##", you can surround it with angle brackets: This will force the parser to consume the whole URI: You can also use create named links, where you choose the displayed text. These work within Phabricator or on the internet at large: [[/herald/transcript/ | Herald Transcripts]] [[http://www.boring-legal-documents.com/ | exciting legal documents]] Markdown-style links are also supported: [Toil](http://www.trouble.com) = Linking to Objects = You can link to Differential revisions, Diffusion commits and Maniphest tasks by mentioning the name of an object: D123 # Link to Differential revision D123 rX123 # Link to SVN commit 123 from the "X" repository rXaf3192cd5 # Link to Git commit "af3192cd5..." from the "X" repository. # You must specify at least 7 characters of the hash. T123 # Link to Maniphest task T123 You can also link directly to a comment in Maniphest and Differential: T123#4 # Link to comment #4 of T123 = Embedding Objects You can also generate full-name references to some objects by using braces: {D123} # Link to Differential revision D123 with the full name {T123} # Link to Maniphest task T123 with the full name These references will also show when an object changes state (for instance, a task or revision is closed). Some types of objects support rich embedding. == Embedding Mocks (Pholio) You can embed a Pholio mock by using braces to refer to it: {M123} By default the first four images from the mock set are displayed. This behavior can be overridden with the **image** option. With the **image** option you can provide one or more image IDs to display. You can set the image (or images) to display like this: {M123, image=12345} {M123, image=12345 & 6789} == Embedding Pastes You can embed a Paste using braces: {P123} You can adjust the embed height with the `lines` option: {P123, lines=15} You can highlight specific lines with the `highlight` option: {P123, highlight=15} {P123, highlight="23-25, 31"} == Embedding Images You can embed an image or other file by using braces to refer to it: {F123} In most interfaces, you can drag-and-drop an image from your computer into the text area to upload and reference it. Some browsers (e.g. Chrome) support uploading an image data just by pasting them from clipboard into the text area. You can set file display options like this: {F123, layout=left, float, size=full} Valid options are: - **layout** left (default), center, right, inline, link (render a link instead of a thumbnail for images) - **float** If layout is set to left or right, the image will be floated so text wraps around it. - **size** thumb (default), full - **name** with `layout=link` or for non-images, use this name for the link text - **width** Scale image to a specific width. - **height** Scale image to a specific height. == Embedding Countdowns You can embed a countdown by using braces: {C123} = Quoting Text = To quote text, preface it with an ">": > This is quoted text. This appears like this: > This is quoted text. = Embedding Media = If you set a configuration flag, you can embed media directly in text: - **remarkup.enable-embedded-youtube**: allows you to paste in YouTube videos and have them render inline. This option is disabled by default because it has security and/or silliness implications. Read the description in ##default.conf.php## before enabling it. = Image Macros = You can upload image macros (More Stuff -> Macro) which will replace text strings with the image you specify. For instance, you could upload an image of a dancing banana to create a macro named "peanutbutterjellytime", and then any time you type that string on a separate line it will be replaced with the image of a dancing banana. = Memes = You can also use image macros in the context of memes. For example, if you have an image macro named "grumpy", you can create a meme by doing the following: {meme, src = grumpy, above = toptextgoeshere, below = bottomtextgoeshere} By default, the font used to create the text for the meme is `tuffy.ttf`. For the more authentic feel of `impact.ttf`, you simply have to place the Impact TrueType font in the Phabricator subfolder `/resources/font/`. If Remarkup detects the presence of `impact.ttf`, it will automatically use it. = Mentioning Users = In Differential and Maniphest, you can mention another user by writing: @username When you submit your comment, this will add them as a CC on the revision or task if they aren't already CC'd. = Phriction Documents = You can link to Phriction documents with a name or path: Make sure you sign and date your [[legal/Letter of Marque and Reprisal]]! With a pipe (##|##), you can retitle the link. Use this to mislead your opponents: Check out these [[legal/boring_documents/ | exciting legal documents]]! = Literal Blocks = To place text in a literal block use "%%%": %%%Text that won't be processed by remarkup [[http://www.example.com | example]] %%% Remarkup will not process the text inside of literal blocks (other than to escape HTML and preserve line breaks). = Tables = Remarkup supports simple table syntax. For example, this: | Fruit | Color | Price | Peel? | ----- | ----- | ----- | ----- | Apple | red | `$0.93` | no | Banana | yellow | `$0.19` | **YES** ...produces this: | Fruit | Color | Price | Peel? | ----- | ----- | ----- | ----- | Apple | red | `$0.93` | no | Banana | yellow | `$0.19` | **YES** Remarkup also supports a simplified HTML table syntax. For example, this:
Fruit Color Price Peel?
Apple red `$0.93` no
Banana yellow `$0.19` **YES**
...produces this:
Fruit Color Price Peel?
Apple red `$0.93` no
Banana yellow `$0.19` **YES**
Some general notes about this syntax: - your tags must all be properly balanced; - your tags must NOT include attributes (`` is OK, `` is not); - you can use other Remarkup rules (like **bold**, //italics//, etc.) inside table cells. = Fullscreen Mode = Remarkup editors provide a fullscreen composition mode. This can make it easier to edit large blocks of text, or improve focus by removing distractions. You can exit **Fullscreen** mode by clicking the button again or by pressing escape. diff --git a/src/infrastructure/markup/PhabricatorMarkupEngine.php b/src/infrastructure/markup/PhabricatorMarkupEngine.php index ff01cb4bc1..5fa43f8f66 100644 --- a/src/infrastructure/markup/PhabricatorMarkupEngine.php +++ b/src/infrastructure/markup/PhabricatorMarkupEngine.php @@ -1,585 +1,586 @@ addObject($comment, $field); * } * * Now, call @{method:process} to perform the actual cache/rendering * step. This is a heavyweight call which does batched data access and * transforms the markup into output. * * $engine->process(); * * Finally, do something with the results: * * $results = array(); * foreach ($comments as $comment) { * $results[] = $engine->getOutput($comment, $field); * } * * If you have a single object to render, you can use the convenience method * @{method:renderOneObject}. * * @task markup Markup Pipeline * @task engine Engine Construction */ final class PhabricatorMarkupEngine { private $objects = array(); private $viewer; private $version = 8; /* -( Markup Pipeline )---------------------------------------------------- */ /** * Convenience method for pushing a single object through the markup * pipeline. * * @param PhabricatorMarkupInterface The object to render. * @param string The field to render. * @param PhabricatorUser User viewing the markup. * @return string Marked up output. * @task markup */ public static function renderOneObject( PhabricatorMarkupInterface $object, $field, PhabricatorUser $viewer) { return id(new PhabricatorMarkupEngine()) ->setViewer($viewer) ->addObject($object, $field) ->process() ->getOutput($object, $field); } /** * Queue an object for markup generation when @{method:process} is * called. You can retrieve the output later with @{method:getOutput}. * * @param PhabricatorMarkupInterface The object to render. * @param string The field to render. * @return this * @task markup */ public function addObject(PhabricatorMarkupInterface $object, $field) { $key = $this->getMarkupFieldKey($object, $field); $this->objects[$key] = array( 'object' => $object, 'field' => $field, ); return $this; } /** * Process objects queued with @{method:addObject}. You can then retrieve * the output with @{method:getOutput}. * * @return this * @task markup */ public function process() { $keys = array(); foreach ($this->objects as $key => $info) { if (!isset($info['markup'])) { $keys[] = $key; } } if (!$keys) { return; } $objects = array_select_keys($this->objects, $keys); // Build all the markup engines. We need an engine for each field whether // we have a cache or not, since we still need to postprocess the cache. $engines = array(); foreach ($objects as $key => $info) { $engines[$key] = $info['object']->newMarkupEngine($info['field']); $engines[$key]->setConfig('viewer', $this->viewer); } // Load or build the preprocessor caches. $blocks = $this->loadPreprocessorCaches($engines, $objects); $blocks = mpull($blocks, 'getCacheData'); $this->engineCaches = $blocks; // Finalize the output. foreach ($objects as $key => $info) { $engine = $engines[$key]; $field = $info['field']; $object = $info['object']; $output = $engine->postprocessText($blocks[$key]); $output = $object->didMarkupText($field, $output, $engine); $this->objects[$key]['output'] = $output; } return $this; } /** * Get the output of markup processing for a field queued with * @{method:addObject}. Before you can call this method, you must call * @{method:process}. * * @param PhabricatorMarkupInterface The object to retrieve. * @param string The field to retrieve. * @return string Processed output. * @task markup */ public function getOutput(PhabricatorMarkupInterface $object, $field) { $key = $this->getMarkupFieldKey($object, $field); $this->requireKeyProcessed($key); return $this->objects[$key]['output']; } /** * Retrieve engine metadata for a given field. * * @param PhabricatorMarkupInterface The object to retrieve. * @param string The field to retrieve. * @param string The engine metadata field to retrieve. * @param wild Optional default value. * @task markup */ public function getEngineMetadata( PhabricatorMarkupInterface $object, $field, $metadata_key, $default = null) { $key = $this->getMarkupFieldKey($object, $field); $this->requireKeyProcessed($key); return idx($this->engineCaches[$key]['metadata'], $metadata_key, $default); } /** * @task markup */ private function requireKeyProcessed($key) { if (empty($this->objects[$key])) { throw new Exception( "Call addObject() before using results (key = '{$key}')."); } if (!isset($this->objects[$key]['output'])) { throw new Exception( "Call process() before using results."); } } /** * @task markup */ private function getMarkupFieldKey( PhabricatorMarkupInterface $object, $field) { static $custom; if ($custom === null) { $custom = array_merge( self::loadCustomInlineRules(), self::loadCustomBlockRules()); $custom = mpull($custom, 'getRuleVersion', null); ksort($custom); $custom = PhabricatorHash::digestForIndex(serialize($custom)); } return $object->getMarkupFieldKey($field).'@'.$this->version.'@'.$custom; } /** * @task markup */ private function loadPreprocessorCaches(array $engines, array $objects) { $blocks = array(); $use_cache = array(); foreach ($objects as $key => $info) { if ($info['object']->shouldUseMarkupCache($info['field'])) { $use_cache[$key] = true; } } if ($use_cache) { try { $blocks = id(new PhabricatorMarkupCache())->loadAllWhere( 'cacheKey IN (%Ls)', array_keys($use_cache)); $blocks = mpull($blocks, null, 'getCacheKey'); } catch (Exception $ex) { phlog($ex); } } foreach ($objects as $key => $info) { // False check in case MySQL doesn't support unicode characters // in the string (T1191), resulting in unserialize returning false. if (isset($blocks[$key]) && $blocks[$key]->getCacheData() !== false) { // If we already have a preprocessing cache, we don't need to rebuild // it. continue; } $text = $info['object']->getMarkupText($info['field']); $data = $engines[$key]->preprocessText($text); // NOTE: This is just debugging information to help sort out cache issues. // If one machine is misconfigured and poisoning caches you can use this // field to hunt it down. $metadata = array( 'host' => php_uname('n'), ); $blocks[$key] = id(new PhabricatorMarkupCache()) ->setCacheKey($key) ->setCacheData($data) ->setMetadata($metadata); if (isset($use_cache[$key])) { // This is just filling a cache and always safe, even on a read pathway. $unguarded = AphrontWriteGuard::beginScopedUnguardedWrites(); $blocks[$key]->replace(); unset($unguarded); } } return $blocks; } /** * Set the viewing user. Used to implement object permissions. * * @param PhabricatorUser The viewing user. * @return this * @task markup */ public function setViewer(PhabricatorUser $viewer) { $this->viewer = $viewer; return $this; } /* -( Engine Construction )------------------------------------------------ */ /** * @task engine */ public static function newManiphestMarkupEngine() { return self::newMarkupEngine(array( )); } /** * @task engine */ public static function newPhrictionMarkupEngine() { return self::newMarkupEngine(array( 'header.generate-toc' => true, )); } /** * @task engine */ public static function newPhameMarkupEngine() { return self::newMarkupEngine(array( 'macros' => false, )); } /** * @task engine */ public static function newFeedMarkupEngine() { return self::newMarkupEngine( array( 'macros' => false, 'youtube' => false, )); } /** * @task engine */ public static function newDifferentialMarkupEngine(array $options = array()) { return self::newMarkupEngine(array( 'differential.diff' => idx($options, 'differential.diff'), )); } /** * @task engine */ public static function newDiffusionMarkupEngine(array $options = array()) { return self::newMarkupEngine(array( )); } /** * @task engine */ public static function getEngine($ruleset = 'default') { static $engines = array(); if (isset($engines[$ruleset])) { return $engines[$ruleset]; } $engine = null; switch ($ruleset) { case 'default': $engine = self::newMarkupEngine(array()); break; case 'nolinebreaks': $engine = self::newMarkupEngine(array()); $engine->setConfig('preserve-linebreaks', false); break; case 'diviner': $engine = self::newMarkupEngine(array()); $engine->setConfig('preserve-linebreaks', false); // $engine->setConfig('diviner.renderer', new DivinerDefaultRenderer()); $engine->setConfig('header.generate-toc', true); break; default: throw new Exception("Unknown engine ruleset: {$ruleset}!"); } $engines[$ruleset] = $engine; return $engine; } /** * @task engine */ private static function getMarkupEngineDefaultConfiguration() { return array( 'pygments' => PhabricatorEnv::getEnvConfig('pygments.enabled'), 'youtube' => PhabricatorEnv::getEnvConfig( 'remarkup.enable-embedded-youtube'), 'differential.diff' => null, 'header.generate-toc' => false, 'macros' => true, 'uri.allowed-protocols' => PhabricatorEnv::getEnvConfig( 'uri.allowed-protocols'), 'syntax-highlighter.engine' => PhabricatorEnv::getEnvConfig( 'syntax-highlighter.engine'), 'preserve-linebreaks' => true, ); } /** * @task engine */ public static function newMarkupEngine(array $options) { $options += self::getMarkupEngineDefaultConfiguration(); $engine = new PhutilRemarkupEngine(); $engine->setConfig('preserve-linebreaks', $options['preserve-linebreaks']); $engine->setConfig('pygments.enabled', $options['pygments']); $engine->setConfig( 'uri.allowed-protocols', $options['uri.allowed-protocols']); $engine->setConfig('differential.diff', $options['differential.diff']); $engine->setConfig('header.generate-toc', $options['header.generate-toc']); $engine->setConfig( 'syntax-highlighter.engine', $options['syntax-highlighter.engine']); $rules = array(); $rules[] = new PhutilRemarkupRuleEscapeRemarkup(); $rules[] = new PhutilRemarkupRuleMonospace(); $rules[] = new PhutilRemarkupRuleDocumentLink(); if ($options['youtube']) { $rules[] = new PhabricatorRemarkupRuleYoutube(); } $applications = PhabricatorApplication::getAllInstalledApplications(); foreach ($applications as $application) { foreach ($application->getRemarkupRules() as $rule) { $rules[] = $rule; } } $rules[] = new PhutilRemarkupRuleHyperlink(); if ($options['macros']) { $rules[] = new PhabricatorRemarkupRuleImageMacro(); $rules[] = new PhabricatorRemarkupRuleMeme(); } $rules[] = new PhutilRemarkupRuleBold(); $rules[] = new PhutilRemarkupRuleItalic(); $rules[] = new PhutilRemarkupRuleDel(); + $rules[] = new PhutilRemarkupRuleUnderline(); foreach (self::loadCustomInlineRules() as $rule) { $rules[] = $rule; } $blocks = array(); $blocks[] = new PhutilRemarkupEngineRemarkupQuotesBlockRule(); $blocks[] = new PhutilRemarkupEngineRemarkupLiteralBlockRule(); $blocks[] = new PhutilRemarkupEngineRemarkupHeaderBlockRule(); $blocks[] = new PhutilRemarkupEngineRemarkupHorizontalRuleBlockRule(); $blocks[] = new PhutilRemarkupEngineRemarkupListBlockRule(); $blocks[] = new PhutilRemarkupEngineRemarkupCodeBlockRule(); $blocks[] = new PhutilRemarkupEngineRemarkupNoteBlockRule(); $blocks[] = new PhutilRemarkupEngineRemarkupTableBlockRule(); $blocks[] = new PhutilRemarkupEngineRemarkupSimpleTableBlockRule(); $blocks[] = new PhutilRemarkupEngineRemarkupInterpreterRule(); $blocks[] = new PhutilRemarkupEngineRemarkupDefaultBlockRule(); foreach (self::loadCustomBlockRules() as $rule) { $blocks[] = $rule; } foreach ($blocks as $block) { $block->setMarkupRules($rules); } $engine->setBlockRules($blocks); return $engine; } public static function extractPHIDsFromMentions(array $content_blocks) { $mentions = array(); $engine = self::newDifferentialMarkupEngine(); $engine->setConfig('viewer', PhabricatorUser::getOmnipotentUser()); foreach ($content_blocks as $content_block) { $engine->markupText($content_block); $phids = $engine->getTextMetadata( PhabricatorRemarkupRuleMention::KEY_MENTIONED, array()); $mentions += $phids; } return $mentions; } public static function extractFilePHIDsFromEmbeddedFiles( array $content_blocks) { $files = array(); $engine = self::newDifferentialMarkupEngine(); $engine->setConfig('viewer', PhabricatorUser::getOmnipotentUser()); foreach ($content_blocks as $content_block) { $engine->markupText($content_block); $ids = $engine->getTextMetadata( PhabricatorRemarkupRuleEmbedFile::KEY_EMBED_FILE_PHIDS, array()); $files += $ids; } return $files; } /** * Produce a corpus summary, in a way that shortens the underlying text * without truncating it somewhere awkward. * * TODO: We could do a better job of this. * * @param string Remarkup corpus to summarize. * @return string Summarized corpus. */ public static function summarize($corpus) { // Major goals here are: // - Don't split in the middle of a character (utf-8). // - Don't split in the middle of, e.g., **bold** text, since // we end up with hanging '**' in the summary. // - Try not to pick an image macro, header, embedded file, etc. // - Hopefully don't return too much text. We don't explicitly limit // this right now. $blocks = preg_split("/\n *\n\s*/", trim($corpus)); $best = null; foreach ($blocks as $block) { // This is a test for normal spaces in the block, i.e. a heuristic to // distinguish standard paragraphs from things like image macros. It may // not work well for non-latin text. We prefer to summarize with a // paragraph of normal words over an image macro, if possible. $has_space = preg_match('/\w\s\w/', $block); // This is a test to find embedded images and headers. We prefer to // summarize with a normal paragraph over a header or an embedded object, // if possible. $has_embed = preg_match('/^[{=]/', $block); if ($has_space && !$has_embed) { // This seems like a good summary, so return it. return $block; } if (!$best) { // This is the first block we found; if everything is garbage just // use the first block. $best = $block; } } return $best; } private static function loadCustomInlineRules() { return id(new PhutilSymbolLoader()) ->setAncestorClass('PhabricatorRemarkupCustomInlineRule') ->loadObjects(); } private static function loadCustomBlockRules() { return id(new PhutilSymbolLoader()) ->setAncestorClass('PhabricatorRemarkupCustomBlockRule') ->loadObjects(); } }