Examples

Here is a simple example PHP script using the tokenizer that will read in a PHP file, strip all comments from the source and print the pure code only.

Example #1 Strip comments with the tokenizer

<?php
$source
= file_get_contents('example.php');
$tokens = token_get_all($source);

foreach (
$tokens as $token) {
if (
is_string($token)) {
// simple 1-character token
echo $token;
} else {
// token array
list($id, $text) = $token;

switch (
$id) {
case
T_COMMENT:
case
T_DOC_COMMENT:
// no action on comments
break;

default:
// anything else -> output "as is"
echo $text;
break;
}
}
}
?>
add a note add a note

User Contributed Notes 1 note

up
0
anant kumar singh
10 years ago
<?php
$fileStr
= file_get_contents('filepath');
$newStr  = '';

$commentTokens = array(T_COMMENT);

if (
defined('T_DOC_COMMENT'))
   
$commentTokens[] = T_DOC_COMMENT; // PHP 5
if (defined('T_ML_COMMENT'))
   
$commentTokens[] = T_ML_COMMENT// PHP 4

$tokens = token_get_all($fileStr);

foreach (
$tokens as $token) {   
    if (
is_array($token)) {
        if (
in_array($token[0], $commentTokens))
            continue;

       
$token = $token[1];
    }

   
$newStr .= $token;
}
$newfile = file_put_contents('newfilename',$newStr);
//echo $newStr;die;
?>
To Top