Login   Register  
PHP Classes
elePHPant
Icontem

File: usage.php

Recommend this page to a friend!
Stumble It! Stumble It! Bookmark in del.icio.us Bookmark in del.icio.us
  Classes of Domenico Pontari  >  Tokenizer  >  usage.php  >  Download  
File: usage.php
Role: Example script
Content type: text/plain
Description: different examples of the usage
Class: Tokenizer
Tokenizer split strings into tokens
Author: By
Last change: fixing an uploding error
Date: 4 years ago
Size: 828 bytes
 

Contents

Class file image Download
<?php
    
require_once ('tokenizer.php');
    
    
/******* TOKENIZER EXAMPLES ********/

    // Simple example
    /*
        [0]=> "Hello"
        [1]=> " "
        [2]=> "world!"
    */
    
$tokenizer tokenizer::registerDefaultTokenizer();
    
var_dump ($tokenizer->tokenize ('Hello world!'));
    
    
// Example with markers
    /*
        [0]=> "Please"
        [1]=> " "
        [2]=> "not"
        [3]=> " "
        [4]=> ""cut quotation points"!"
    */
    
var_dump ($tokenizer->tokenize ('Please not "cut quotation points"!'));

    
// Example with escaping char
    /*
        [0]=> "Mark"
        [1]=> " "
        [2]=> "said:"
        [3]=> " "
        [4]=> "'where is Janet\'s jacket'?"
    */
    
var_dump ($tokenizer->tokenize ("Mark said: 'where is Janet\'s jacket'?"));
?>