=== Berq Markdown: Markdown Pages for AI & LLM Crawlers ===
Contributors: berqwp, thevisionofhamza
Tags: markdown, llm, ai, seo, crawler
Requires at least: 6.6
Tested up to: 7.0
Requires PHP: 7.4
Stable tag: 1.3.2
License: GPL-2.0-or-later
License URI: https://www.gnu.org/licenses/gpl-2.0.html

Increase your AI visibility. Berq Markdown serves every page as a clean Markdown file so AI assistants and LLM crawlers can read, cite, and recommend your content.

== Description ==

AI assistants and LLM crawlers (GPTBot, ClaudeBot, Googlebot-AI) struggle to read messy HTML, and content they cannot read cleanly is content they cannot cite. Berq Markdown exists to fix that and increase your AI visibility. It gives every page on your site a clean, structured Markdown version, so your content is more likely to be read accurately, cited correctly, and recommended by AI tools instead of getting skipped or misread. It does all of this without duplicating your content or creating any separate pages to manage.

Append `.md` to any public page or post URL (for example `https://example.com/about.md`) and you get a clean, structured Markdown file with YAML frontmatter, ready for AI tools to consume directly. The plugin handles content from all common sources: Gutenberg blocks, classic editor, shortcodes, and custom PHP templates, so your AI visibility gain covers your entire site, not just pages built one specific way.

**How it works**

When a visitor or crawler requests `/about.md`, the plugin intercepts the request, finds the matching WordPress post, converts the HTML content to Markdown, and serves it with the correct `Content-Type: text/markdown` header, the format AI tools expect. No new database entries or files are created, so you gain AI visibility without adding any clutter to your site.

**Key Features**

* **Automatic .md URLs**: any public post or page URL plus `.md` returns clean Markdown, so AI crawlers get a direct, accurate read of your content instead of a messy HTML parse, increasing the odds your content gets cited.
* **YAML frontmatter**: every .md file includes `title`, `description`, `url`, `date_modified`, `author`, and `language`, so AI tools can attribute and date your content correctly instead of guessing, which improves how confidently they cite you.
* **Full content compatibility**: works with Gutenberg blocks, classic editor, shortcodes, and custom PHP page templates, so your AI visibility gain reaches every page on your site, not just a few.
* **HTML to Markdown conversion**: pure PHP with no external dependencies. It handles headings, paragraphs, lists, links, images, tables, and code blocks, so AI tools get well formed Markdown instead of a broken or partial conversion.
* **`<link>` tag injection**: injects a `<link rel="alternate" type="text/markdown">` tag in every page's `<head>`, so crawlers can discover the .md version of every page on their own, increasing the chance they read the clean version instead of the raw HTML.
* **Multilingual support**: detects language via Polylang or WPML, adds `language` and `alternates` fields to the frontmatter, and emits `hreflang` link tags, so AI tools recommend the correct language version of your content to the right audience instead of the wrong one.
* **Post type selection**: choose which post types (posts, pages, custom post types) are served as Markdown, so you decide exactly which content is eligible to be read and cited by AI tools.
* **URL exclusions with wildcards**: exclude specific URL patterns using `*` as a wildcard (for example `/cart/*`, `*/checkout*`), so private or transactional areas like checkout and account pages are kept out of AI crawling entirely.
* **Transient caching**: optional cache with a configurable TTL that absorbs traffic spikes from AI crawlers, so your site stays fast for human visitors even under heavy bot traffic.
* **`[btrmd313_md_link]` shortcode**: renders a link to the current page's `.md` file and opens it in a new tab, giving readers an easy way to grab a clean copy of your content for their own AI tools.
* **Public content rendering**: always renders content as a non logged in visitor, so member only sections are never exposed through the Markdown endpoint.
* **Admin settings page**: every option is configurable under Settings > Berq Markdown, with a Settings link on the Plugins page, so you stay in control without touching code.

== Other Products ==

Also check out our other WordPress products.

* [BerqWP](https://berqwp.com/)
* [WP Replai](https://wpreplai.com/)
* [Berq Used CSS](https://wordpress.org/plugins/berq-used-css/)

== Installation ==

1. Upload the `berq-markdown` folder to `/wp-content/plugins/`
2. Activate the plugin through the **Plugins** menu in WordPress
3. Go to **Settings > Permalinks** and click **Save Changes** to flush rewrite rules (required once after activation)
4. Visit any page URL with `.md` appended, for example `https://yoursite.com/about.md`
5. Optionally configure the plugin under **Settings > Berq Markdown**

== Frequently Asked Questions ==

= Does this create duplicate content? =

No. The `.md` URLs are separate endpoints that return `Content-Type: text/markdown`, so search engines treat them differently from HTML pages. The plugin also sends `X-Robots-Tag: noindex, follow` by default, which keeps the Markdown versions out of the search index while still letting AI crawlers follow and read them, so you gain AI visibility without any SEO downside.

= Will it work with my page builder? =

Yes. The plugin renders content the same way WordPress does for a logged out visitor, using the full `the_content` filter chain, which covers Gutenberg blocks and most page builders. For pages using custom PHP templates with no `post_content`, it falls back to output buffering the template and extracting the body content, so even unusual page setups still get a working `.md` version.

= How do I exclude certain pages from .md serving? =

Go to **Settings > Berq Markdown > URL Exclusions** and enter one URL pattern per line. Use `*` as a wildcard. Examples: `/cart/*`, `*/checkout*`, `/my-account/`. This keeps private or transactional pages out of AI crawlers entirely.

= Does it work with Yoast SEO? =

Yes. The plugin reads Yoast's meta description and uses it for the frontmatter `description` field, so the description AI tools see matches the one you already wrote for SEO, keeping your AI visibility and search visibility consistent.

= Does it support custom post types? =

Yes. Go to **Settings > Berq Markdown > Post Types** and check any registered public post type you want included, so you can extend AI friendly content beyond just posts and pages.

= Is caching enabled by default? =

No, caching is disabled by default so you always see fresh content during development. Enable it under **Settings > Berq Markdown > Cache** when you are ready to scale and want to reduce server load from repeated crawler requests.

= What shortcode is available? =

`[btrmd313_md_link]` outputs a link to the current page's `.md` version and opens it in a new tab.

Optional attributes:
* `text`: link label (default: "View as Markdown")
* `class`: additional CSS class or classes

Example: `[btrmd313_md_link text="Read as Markdown" class="my-button"]`

= Does it support multilingual sites? =

Yes. The plugin detects language via Polylang or WPML. It adds a `language` field to the frontmatter and an `alternates` block listing all translated `.md` URLs. It also emits `<link rel="alternate" hreflang="...">` tags in the HTML `<head>`, so AI tools and crawlers always land on the right language version of your content.

= Does it work with caching plugins? =

Yes. The plugin serves `.md` files before WordPress loads any theme template. Cache plugins that operate at the HTTP level, such as full page caching, may cache `.md` responses too. That's fine and actually helps performance under heavy crawler traffic.

== Screenshots ==

1. Settings page showing General, Post Types, Exclusions, Cache, Headers, and Advanced tabs

== Changelog ==

= 1.3.2 =
* Set max 5 minutes cache age limit for .md endpoints.

= 1.2.0 =
* Removed: Automatic internal link rewriting to `.md` equivalents. Links inside Markdown output now keep their original URLs.
* Removed: Sitemap integration (Yoast provider and the standalone `/md-sitemap.xml` endpoint).
* Change: `.md` responses now send a fixed `Cache-Control: public, max-age=300` header, so browsers and CDNs cache them for a short, predictable window regardless of the server side cache TTL setting.

= 1.1.0 =
* Fix: MD files now served with correct 200 HTTP status (WordPress was setting 404 before template_redirect fired)
* Fix: Home page `.md` URL corrected to `/index.md` instead of broken `site.local.md`
* Fix: `url_to_postid()` trailing slash sensitivity. The plugin now tries the URL both with and without a trailing slash
* Fix: PHP template pages with empty `post_content` now rendered correctly via output buffer
* Fix: HTML comments (including Gutenberg block delimiters) stripped from Markdown output
* Fix: Heading whitespace normalized. Multi line headings now collapse to a single line
* Fix: Inline tags inside headings stripped to plain text
* Fix: Description frontmatter field no longer wrapped in unnecessary quotes
* Fix: Cache disabled by default, opt in instead of opt out
* Fix: WordPress Plugin Check compliance. Text domain corrected, `parse_url` switched to `wp_parse_url`, `strip_tags` switched to `wp_strip_all_tags`, variable prefixing fixed in uninstall
* Feature: Post type selection. Choose which post types are served as `.md` files
* Feature: Multilingual support with `language` and `alternates` frontmatter fields and `hreflang` link tags (Polylang and WPML)
* Feature: Internal links inside Markdown output rewritten to `.md` equivalents automatically
* Feature: `[btrmd313_md_link]` shortcode renders a link to the current page's `.md` file
* Feature: Template pages rendered as non logged in visitor for accurate public content
* Feature: Admin bar suppressed during template output buffering

= 1.0.0 =
* Initial release
* .md URL serving with YAML frontmatter
* HTML to Markdown conversion (pure PHP, DOMDocument)
* Gutenberg block, shortcode, and custom template support
* URL exclusions with wildcard support
* Post type selection
* Transient cache with configurable TTL
* `<link rel="alternate">` injection in HTML head
* `[btrmd313_md_link]` shortcode
* Yoast SEO sitemap integration plus generic md-sitemap.xml fallback
* Multilingual support (Polylang, WPML) with language frontmatter and hreflang tags
* Internal link rewriting to .md equivalents
* Non logged in rendering for accurate public content
* Admin settings page under Settings > Berq Markdown

== Upgrade Notice ==

= 1.0.0 =
Initial release. After activation, visit Settings > Permalinks and click Save to flush rewrite rules.
