When the documentation says that the PHP parser ignores everything outside the <?php ... ?> tags, it means literally EVERYTHING. Including things you normally wouldn't consider "valid", such as the following:
<html><body>
<p<?php if ($highlight): ?> class="highlight"<?php endif;?>>This is a paragraph.</p>
</body></html>
Notice how the PHP code is embedded in the middle of an HTML opening tag. The PHP parser doesn't care that it's in the middle of an opening tag, and doesn't require that it be closed. It also doesn't care that after the closing ?> tag is the end of the HTML opening tag. So, if $highlight is true, then the output will be:
<html><body>
<p class="highlight">This is a paragraph.</p>
</body></html>
Otherwise, it will be:
<html><body>
<p>This is a paragraph.</p>
</body></html>
Using this method, you can have HTML tags with optional attributes, depending on some PHP condition. Extremely flexible and useful!