From 5637a66459b96c27066eb86ad38dc948d383e34d Mon Sep 17 00:00:00 2001 From: pauliyobo Date: Fri, 5 Mar 2021 21:11:29 +0100 Subject: [PATCH 1/4] hopefully made the documentation more clearer on what concerns preprocessor implementation with non rust languages --- guide/src/for_developers/preprocessors.md | 33 ++++++++++++++++++++--- 1 file changed, 30 insertions(+), 3 deletions(-) diff --git a/guide/src/for_developers/preprocessors.md b/guide/src/for_developers/preprocessors.md index c8086298..5acf29a9 100644 --- a/guide/src/for_developers/preprocessors.md +++ b/guide/src/for_developers/preprocessors.md @@ -34,8 +34,8 @@ command = "python3 /path/to/foo.py" renderer = ["html", "epub"] ``` -In typical unix style, all inputs to the plugin will be written to `stdin` as -JSON and `mdbook` will read from `stdout` if it is expecting output. +Once the preprocessor has been defined and the build process has started, MdBook will execute the command defined in the `preprocessor.foo.command` key passing the arguments support and the renderer name, monitoring the status code of the executed command. +If the status code retrieved is 0, the library will be sending through stdin both the context and the book representation serialized in JSON format, and it'll be capturing the response from stdout, which will be the modified book which has to also be serialized in json format. The easiest way to get started is by creating your own implementation of the `Preprocessor` trait (e.g. in `lib.rs`) and then creating a shell binary which @@ -80,7 +80,7 @@ without accidentally breaking the document. ```rust fn remove_emphasis( num_removed_items: &mut usize, - chapter: &mut Chapter, + chapter: &mut Chapter, ) -> Result { let mut buf = String::with_capacity(chapter.content.len()); @@ -106,6 +106,33 @@ fn remove_emphasis( For everything else, have a look [at the complete example][example]. +## implementing a preprocessor with a different language +The fact that MdBook utilizes stdin and stdout to communicate with the preprocessors, makes it easy for developers to implement them in a language different than rust. +The following code shows how to implement a simple preprocessor in python, which will modify the content of the first chapter. +The example will follow the configuration shown above with `preprocessor.foo.command` actually pointing to a python script. The code of said script is below: + +```python +import json +import sys + + +if __name__ == '__main__': + if len(sys.argv) > 1: # we check if we received any argument + if sys.argv[1] == "supports": + # then we are good to return an exit status code of 0, since the other argument will just be the renderer's name + sys.exit(0) + + # we will load both the context and the book representations from stdin + stdin = sys.stdin + context, book = json.load(stdin) + # and now, we can just modify the content of the first chapter + book['sections'][0]['Chapter']['content'] = '

Hello

' + # we are done with the book's modification, we can just print it to stdout, + print(json.dumps(book)) +``` + + + [preprocessor-docs]: https://docs.rs/mdbook/latest/mdbook/preprocess/trait.Preprocessor.html [pc]: https://crates.io/crates/pulldown-cmark [pctc]: https://crates.io/crates/pulldown-cmark-to-cmark From dcccd3289d4e03d98e0364b80cb4992d83c51eca Mon Sep 17 00:00:00 2001 From: Paul <34284755+pauliyobo@users.noreply.github.com> Date: Mon, 8 Mar 2021 20:39:39 +0100 Subject: [PATCH 2/4] Apply suggestions from code review improved readability and cleared sections which could have caused more confusion Co-authored-by: Eric Huss --- guide/src/for_developers/preprocessors.md | 15 +++++++++++---- 1 file changed, 11 insertions(+), 4 deletions(-) diff --git a/guide/src/for_developers/preprocessors.md b/guide/src/for_developers/preprocessors.md index 5acf29a9..40c94ce4 100644 --- a/guide/src/for_developers/preprocessors.md +++ b/guide/src/for_developers/preprocessors.md @@ -34,8 +34,15 @@ command = "python3 /path/to/foo.py" renderer = ["html", "epub"] ``` -Once the preprocessor has been defined and the build process has started, MdBook will execute the command defined in the `preprocessor.foo.command` key passing the arguments support and the renderer name, monitoring the status code of the executed command. -If the status code retrieved is 0, the library will be sending through stdin both the context and the book representation serialized in JSON format, and it'll be capturing the response from stdout, which will be the modified book which has to also be serialized in json format. +Once the preprocessor has been defined and the build process starts, mdBook executes the command defined in the `preprocessor.foo.command` key twice. +The first time it runs the preprocessor to determine if it supports the given renderer. +mdBook passes two arguments to the process: the first argument is the string `supports` and the second argument is the renderer name. +The preprocessor should exit with a status code 0 if it supports the given renderer, or return a non-zero exit code if it does not. + +If the preprocessor supports the renderer, then mdbook runs it a second time, passing JSON data into stdin. +The JSON consists of an array of `[context, book]` where `context` is the serialized object [`PreprocessorContext`] and `book` is a [`Book`] object containing the content of the book. + +The preprocessor should return the JSON format of the [`Book`] object to stdout, with any modifications it wishes to perform. The easiest way to get started is by creating your own implementation of the `Preprocessor` trait (e.g. in `lib.rs`) and then creating a shell binary which @@ -107,7 +114,7 @@ fn remove_emphasis( For everything else, have a look [at the complete example][example]. ## implementing a preprocessor with a different language -The fact that MdBook utilizes stdin and stdout to communicate with the preprocessors, makes it easy for developers to implement them in a language different than rust. +The fact that mdBook utilizes stdin and stdout to communicate with the preprocessors makes it easy to implement them in a language other than Rust. The following code shows how to implement a simple preprocessor in python, which will modify the content of the first chapter. The example will follow the configuration shown above with `preprocessor.foo.command` actually pointing to a python script. The code of said script is below: @@ -126,7 +133,7 @@ if __name__ == '__main__': stdin = sys.stdin context, book = json.load(stdin) # and now, we can just modify the content of the first chapter - book['sections'][0]['Chapter']['content'] = '

Hello

' + book['sections'][0]['Chapter']['content'] = '# Hello' # we are done with the book's modification, we can just print it to stdout, print(json.dumps(book)) ``` From 1a2fa292093678ffcaa60348618d0037167e6fcd Mon Sep 17 00:00:00 2001 From: pauliyobo Date: Sat, 4 Sep 2021 23:44:27 +0200 Subject: [PATCH 3/4] introduced proposed suggestions related to the documentation --- guide/src/for_developers/preprocessors.md | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/guide/src/for_developers/preprocessors.md b/guide/src/for_developers/preprocessors.md index 40c94ce4..7e93c337 100644 --- a/guide/src/for_developers/preprocessors.md +++ b/guide/src/for_developers/preprocessors.md @@ -87,7 +87,7 @@ without accidentally breaking the document. ```rust fn remove_emphasis( num_removed_items: &mut usize, - chapter: &mut Chapter, + chapter: &mut Chapter, ) -> Result { let mut buf = String::with_capacity(chapter.content.len()); @@ -115,8 +115,8 @@ For everything else, have a look [at the complete example][example]. ## implementing a preprocessor with a different language The fact that mdBook utilizes stdin and stdout to communicate with the preprocessors makes it easy to implement them in a language other than Rust. -The following code shows how to implement a simple preprocessor in python, which will modify the content of the first chapter. -The example will follow the configuration shown above with `preprocessor.foo.command` actually pointing to a python script. The code of said script is below: +The following code shows how to implement a simple preprocessor in Python, which will modify the content of the first chapter. +The example below follows the configuration shown above with `preprocessor.foo.command` actually pointing to a Python script. ```python import json @@ -129,9 +129,8 @@ if __name__ == '__main__': # then we are good to return an exit status code of 0, since the other argument will just be the renderer's name sys.exit(0) - # we will load both the context and the book representations from stdin - stdin = sys.stdin - context, book = json.load(stdin) + # load both the context and the book representations from stdin + context, book = json.load(sys.stdin) # and now, we can just modify the content of the first chapter book['sections'][0]['Chapter']['content'] = '# Hello' # we are done with the book's modification, we can just print it to stdout, From 656a1825cc37ffd3a3c4493d936c8251a2889845 Mon Sep 17 00:00:00 2001 From: Eric Huss Date: Sun, 5 Sep 2021 18:45:37 -0700 Subject: [PATCH 4/4] Minor fixes for preprocessor docs. --- guide/src/for_developers/preprocessors.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/guide/src/for_developers/preprocessors.md b/guide/src/for_developers/preprocessors.md index 7e93c337..792b2f44 100644 --- a/guide/src/for_developers/preprocessors.md +++ b/guide/src/for_developers/preprocessors.md @@ -113,7 +113,8 @@ fn remove_emphasis( For everything else, have a look [at the complete example][example]. -## implementing a preprocessor with a different language +## Implementing a preprocessor with a different language + The fact that mdBook utilizes stdin and stdout to communicate with the preprocessors makes it easy to implement them in a language other than Rust. The following code shows how to implement a simple preprocessor in Python, which will modify the content of the first chapter. The example below follows the configuration shown above with `preprocessor.foo.command` actually pointing to a Python script. @@ -146,3 +147,5 @@ if __name__ == '__main__': [an example no-op preprocessor]: https://github.com/rust-lang/mdBook/blob/master/examples/nop-preprocessor.rs [`CmdPreprocessor::parse_input()`]: https://docs.rs/mdbook/latest/mdbook/preprocess/trait.Preprocessor.html#method.parse_input [`Book::for_each_mut()`]: https://docs.rs/mdbook/latest/mdbook/book/struct.Book.html#method.for_each_mut +[`PreprocessorContext`]: https://docs.rs/mdbook/latest/mdbook/preprocess/struct.PreprocessorContext.html +[`Book`]: https://docs.rs/mdbook/latest/mdbook/book/struct.Book.html