Archive for the ‘Web Architecture’ Category
Choosing a PHP MVC Framework: Yii Vs. Kohana Vs. Home made MVC
Choosing a framework is never an easy task, there are many aspects to consider, the three most important ones are, from my point of view:
- Available features and community extensions.
- A good documentation with examples.
- A low footprint.
The balance between these three values and the impact of each one on your project will determine the good candidate. Since having a wide range of included features and an active community who provides with extensions, we are sure that the wheel won’t be reinvented. Thus, we can become productive much faster.
ActionScript 3 and cross-domain problem
One of the most typical problems when coding AS3 to, let’s say, read data from an external XML source, is the cross-domain problem. According to security policies, not an url outside the domain where the AS application is being executed can be read, and different subdomains (such as www.yourdomain.com and yourdomain.com) are considered as different ones.
If the external resources you are trying to use come from your own application in another domain / server that you manage, then it can be fixed by adding a cross-domain policy file. This file must be defined on the content source server, and your AS application has to explicitly declare this server as allowed.
Security.allowDomain("domainone.com"); Security.allowDomain("domaintwo.com"); |
But if this is not the case and the content source is on a different remote server like, for instance, when implementing an RSS news reader which grabs content from sources like online news sites, then this problem can be worked out with a PHP proxy file.
Basic web mapper
Sometimes it is useful to have an automated tool to get the full web map of your site. Perhaps not your own web site, since you have already implemented some kind of automatic generation and notification to Google (have not yet?), but a client’s one.
There are a few tools to map an external web site, I tried some in my particular case. They were just adware, or demos, or they obscured the links in the final report… Yeah, of course, sometimes a $30 license is worth it, but you might not want to acquire a new piece of proprietary software every time you need a new feature, might you?
So I decided to write it myself in PHP, not for the money, but for the fun 
Read the rest of this entry »
Scrum / agile development
Some resources and useful information about this development technique:
- About Scrum (Spanish) http://www.proyectosagiles.org/
- Agile Software Development http://www.agile-software-development.com/
- Good Agile Project Management software, community / free version available http://www.rallydev.com/
- Scrum project management software, for Windows (Desktop) http://www.scrumdesk.com
- MoSCoW method to prioritize tasks http://en.wikipedia.org/wiki/MoSCoW_Method
MySQL clean reinstall
To perform a clean reinstall of the MySQL server (which regenerates start scripts, databases and config files) use:
1 2 3 4 | sudo apt-get install mysql-server</li> sudo apt-get remove --purge mysql-server</li> sudo apt-get autoremove mysql-server</li> sudo apt-get install mysql-server</li> |
Some times is useful!
FTP server on Ubuntu
I write this post more as a reminder for myself than as a generic tutorial. Anyway, it is about a precise FTP server configuration on Ubuntu that might or might not be useful for other people. It is just useful in the environment I am using (web server, just one Unix user, some FTP accounts). Here we go:
The server that I’ve used in several cases, and which I prefer just because it is simple enough to have it running quickly, is VSFTP (Very Simple FTP). It works in integration with the machine’s users so that we only need to add users to the server in order to have them available for the FTP daemon. Once the server is installed, a few configurations must be done. In file /etc/vsftpd.conf:
Implementing a database cubes system on MySQL
I’ve already talked about how I solved the problem of managing huge amounts of data in my last post. Now, I’m going to explain how to implement one of the solutions found in order to comfortably face this continuously increasing avalanche of information.
Let’s imagine that, as I explained before, I have separated data input into tables coded by a station ID. All of them are stored in the same database for maintenance simplicity sake. So, we have the ‘database’ in which there are hundreds of tables called ‘Station_XYZ’. Every table has the same structure: to simplify, SensorID, Time (UNIX), Value. All right then, time to make cubes!
Managing big data sets with MySQL
I’m currently working on a database which stores, literally, a million of new data registers every day. They come from several hundred weather stations spread along the world, and need to be processed on real time. That’s a huge amount to be managed with a simple – but powerful – MySQL server, so I’ve had to design and implement a few tricks in order to succeed.
In this post I’d like to talk about the main three techniques we’ve used to engage our high performance database, and the reasons to choose these solutions.
First of all, MySQL innodb tables start degrading at some million registers (7-10M on my current server, but not tested further), so the first though I had was: we need to divide if we want to conquer. Then, after some thinking about, I decided to split the big table into small chunks, and use a table for each station. That made the whole problem more affordable, and tables smaller enough not to become scary: the new problem was how to store about 5 M registers per year per table in our database.
Of course, some logic in both the application controller and the data input server was needed to correctly place data registers. Additionally, tables get often fragmented so some cron jobs were needed to keep the database running smoothly. But there was still the problem of data growing. As long as registers were separated depending on the station, the scalability problem was solved: more tables can be added, even more servers with ranges of stations and a hash function to calculate what server is a given station hosted in. Once again were we talking about several million registers per year per table, and with a standart MySQL 5.1 server (the main problem with an start-up company tends to be funding, so we didn’t have anything better).
Tunneling a multipart POST message through PHP and CURL
In this post I’m going to cover a particular solution to a given problem we’ve faced up in our last project. Let’s set the background: there is a complex web application under development on a shared virtual machine, installed on a subdomain of a public network that will change when going to production. We have also a commercial website hosted in a shared server under a domain that should not change, since it is our first step out of this virtual farm.
For this project, we require to integrate some cameras which send pictures via GPRS connections to the server, and we want to provide some user authentication to avoid having anybody posting whatever to this web service. So, we have developed a controller in Code Igniter which gets a camera ID / password from POST, and also the bytes for the image, and proceeds to do some filtering. For instance, camera ID and password should match with the values previously registered in the database, and the picture should be MIME image/jpeg, with a limit of 2MBs. Up to now, nothing special, tested and working with POSTS from the within the website.
The problem came when we found out that cameras only allow to send their information to a root domain, to a particular file, let’s say upload.php. So, we would only need to put this file in the root path and it’s done. Unfortunately, doing so in the application virtual server would result on flashing again all the cameras once we move to a new server and set up a final domain name. We might be able to install this receiver in the commercial website root path, but then we would require to send the message back to the application server in any way, so it is not the best solution we want to solve this issue.
‘Just add a domain that points to this subdomain and you are done’ would you say. Well, yes, that would have done the trick. Unfortunately, we cannot host this domain in the machine since we have no administrator rights on the servers farm. Nor can we just use this external domain because it produces an HTTP redirection which, to make things worse, is not supported by the camera. We’ve even tried with cloaking the url of the receiver with an external subdomain, with no success – since cloaking wraps the real content inside an iframe and it breaks the POST message.
What a headache! it seemed there was no way to implement the system with the full functionality we wanted…
Managing a web development project
There are many types of websites, as well as sizes and complexity. At the beginning, most of my creations were constructed ‘on the fly’, without much planning – in the end, a ‘news’ module is not scary enough to need a project.
But recently I’ve started working on more ambitious jobs (like http://climaps.com), and feeling the need of a strongly defined method to apply to web applications design. Some tools are almost mandatory, like Use Case Diagrams which become really helpful in order to translate what every user can do into ACL‘s, methods, controllers, etc; or like modeling tools to graphically represent the application’s database.
But the best way – according to my experience – to successfully manage a web project is following a detailed planning. Determining what the steps are and what order should they be executed in, is a good starting point to know were exactly we are on each development stage.
