Category Archives: PHP

Ideas on Code Strategies

This past week I have spent time thinking on some of the best applications I have built, and some of the “neediest” applications I have built. Some of them are in both columns. And I am sure this is the case with most developers. Some times, we get to be part of a major project, and get the best possible ideas, and then BAM, a product is released. After patting ourselves on the back and thinking of how great it is, we find special “features” that require updates or bug fixes. Had we just followed some simple steps during the build phases, I am sure we could have prevented that.

In today’s software development world, many phrases are thrown out there and used. We have SDLC, Agile, Waterfall, Extreme, Scrum, Feature Driven, Test Driven, etc. All of which are great when actually used. But how many places that use those terms are actually doing those practices? There seems to be a bigger practice out there, one that is used more frequently than anyone dares admit. This is a practice I refer to as the Atomic Development Cycle (ADC). I am sure we all have done this, either in its entirety or used many pieces of this.

Characteristics of the ADC can cover a wide range that seem reasonable and responsible, but fall far from it. The project is defined in generalities, and possibly some specifics. The design has been kicked around, maybe even pre-approved. The main data has been identified but not analyzed. The use cases have been discussed, not documented. Timelines are unbelievably tight but manageable. Code is dived into, and testing plans not thought of. The final product looks brilliant seems like flash of light. And it usually is just that, a flash of light, and not sustained. Soon, bugs start appearing. Enhancements require more fixing than actual functional coding. The database design turns out to be too strict and not flexible to the changes, so now Db design changed force more testing and bug fixes. Soon, band-aids are applied to the app, and the app is wrapped up in kluge code and an embarrassment. But since the business saw the initial brilliance, it still wants to use it, and now you are forced to go back, triage and rebuild.

Some projects force my hand into doing this, Other times, it could just be laziness or lack of caring about a personal project that would force my hand in this. With this new year, I need to be better about this. I know better. I know that not all apps are going to work well out of the box forever. But I do know that apps that follow a structured process that ensure the proper framework is in place will succeed more than the ones slopped together. Understanding the proper requirements of the application, even if they do change, is important. Getting the Database in order is tantamount to success. Designing the code is another gigantic step in the right direction. I need to be better and not let this get out of hand.

Yes, the ADC may work in some instances. And yes, they may provide for some awe-inspiring, mind blowing apps. But just like the atomic bombs, they may look brilliant for a short time, but the devastation they leave behind is never worth it.

SOAP Server and Client, now with WSDL part 2

Now here is the part where I give the example files. After we have planned the application functions, we need to figure out what we are going to return to the client. This is going to be a standard array with at least two levels: The Response array and the Data array. It will look similar to this:

Array
(
    [responseMsg] => Array
        (
            [status] => ok
            [message] => Service call was successful
        )
    [allColors] => Array
        (
            [0] => blue
            [1] => green
            [2] => black
            [3] => white
            [4] => yellow
            [5] => red
            [6] => beige
        )
)

Now that we have the basic idea, we need to create the WSDL file. Remember, it is very important to think of WSDL files as of reading from bottom to top. The final WSDl file is located here. Here is the basic idea of the WSDL file I created (going from the bottom to the top):

  1. Service: This houses the binding, the location, the port, and the name.
  2. Binding: This houses the functions that will be exposed, the operation and the input and output encoding. Most of the time these will be similar with only the names being different.
  3. Port Type: Here is where I define the operations and the input/output definitions
  4. Message: These are individual nodes for the Request and Response messages for all functions. These will usually have two message nodes per function, and they will define the structure for each action
  5. Types: This defines each structure that has been mentioned in the Message and any subsequent structures that have to be defined. This is usually the area where most struggles occur.
    • This structure will be encompassed by schema target namespace
    • Import the XML namespaces to help build the structures in the reasponse
    • For each complex type, it should either mention a specific data type (xsd:), or a new defined structure (tns:)
    • Each structure that is an array should be defined as a SOAP-ENC:Array with a wsdl:arrayType parameter

So that is the WSDL. The one I have created defines the 2 functions, the input, the output, and the structure of each. Now we can move on to the Server code.
Continue reading SOAP Server and Client, now with WSDL part 2

SOAP Server and Client, now with WSDL

Trying to get the post in for the day, and this is about the PHP SOAP service. So we have the array of items we need, and we are going to create some quick code for the server and client, but unlike nuSOAP, we do not get a great WSDL generator. So we have to create that ourselves. And it has to work. So that is going to have to work well for us. I have the base WSDL file ready. And it is late tonight, and I got to get a post up, so here is the link to my SOAP directory, and tomorrow we will get into the code and WSDL.

Part 2, with the code

PHP Web Service Example Set Up

In a previous few posts, I posted examples for the NuSOAP server. Now it is time for the intrinsic PHP SOAP web service. This will be a quick post for the server and client, as they are easy to do. The hard part will be the WSDL document. With NuSOAP, it created a WSDL for you, but with PHP SOAP, you must do the WSDL yourself. Be sure to read up on the documentation at PHP.net.

So first off, we need to create a few ideas on what we will return. This will be a simple example with data returned. So for the data set, we will return an array of colors, and an array of named pairs for boys/names and girls/names. Two simple methods to the service, and we need to build the client as well.

So this post will only be the set up of those, and maybe next week I will post the actual server and client code. So for now, here are the data arrays we are going to work with:

$colors = array(
	'blue', 
	'green',
	'black'
	'white',
	'yellow'
	'red',
	'beige'
);

$names = array(
	'boys' => array(
		'stephen',
		'dave',
		'ryan',
		'brian',
		'chris',
		'tom',
	), 
	'girls' => array(
		'elise',
		'sheri',
		'kim',
		'marci',
		'megan',
	)
);

And we will need the WSDL file in place. So the basic layout for this webservice server will act like this:
1. All calls to the colors function will not take any type of parameters, it will be:

$webservice->__soapCall('getColors')

2. Calls to the names function can be one of three options: both, girls, boys:

$webservice->__soapCall('getNames', array('boys'))

3. Both functions will return some type of message header and array of returned data.
4. The data is set up above, and the structure will be the same.

Now knowing this, if we get the WSDL set up and working, then the service will work. So next week, I will show the WSDL, the Server and the Client.

Part 2 including the code

Count the Number of Cakes – Finding complex results with CakePHP

CakePHP offers a good selection of tools to help you retrieve the data. Recently, I came into a situation where I needed to find and paginate results based on a single, distinct column in the table. Distinct data can be tricky, especially if the tools do not allow you to select the distinct based on a column. Distinct will check all columns returned, and coupling in time stamps, 99% of the time all rows will be distinct. So how do you grab the data? Well, first lets examine the sample data that is needed to be extracted first.

The sample data is in a MS SQL Server database. The table contains a record ID, title id, author id, genre, type, last check out date, and edit date. It is possible to have duplicate title, author IDs in the table. We need to extract all DISTINCT title IDs, along with the other information listed where the type is not a paperback, and provide a paginated list. I am sure this would be better architected if needed in the real world. Paginate will only get us so far, as this would only show all records.

class BooksController extends AppController {
    var $paginate = array(
        'order'        => array('Book.id' => 'desc'),
        'fields'    => array('Book.id', 'Book.title_id', 'Book.author_id', 'Book.genre_id', 'Book.type', 'Book.check_date', 'Book.edit_date'),
        'limit'        => 15,
    );
    . . . 
    function index(){
        $this->set('hardback_books', $this->paginate());
    }
}

We need to use more to build a conditional query so the paginate will query against this. We can use CakePHP’s data source to help in this. Now, we could also just write this query out ourselves, but this is helpful to know so when you have to build sub-queries for other items. All data is in MS SQL Server, and we can use normal SQL expressions, but we need to grab DISTINCT data, which goes by rows, not columns, which means we will need to do 2 sub-queries in addition to the main one. So we first need to grab a list of the TOP 1 items. This will be our inner query.

        SELECT TOP 1 * 
        FROM [books] AS [bk_inner] 
        WHERE 
            [bk_inner].[title_id] = [Book].[title_id] 
            AND 
            [bk_inner].[type] <> 'paperback' 

Next, we need to encapsulate that query with an outer one that will select all items which match up to the main query ID.

    SELECT * FROM 
    (
        SELECT TOP 1 * 
        FROM [books] AS [bk_inner] 
        WHERE 
            [bk_inner].[title_id] = [Book].[title_id] 
            AND 
            [bk_inner].[type] <> 'paperback' 
    ) AS [bk_outer] 
    WHERE bk_outer.[title_id] = Book.[title_id] 

So we have the queries, and it needs the main query needs to constrain the results that exists in the sub-queries.

SELECT TOP 15 
    [Book].[id],
    [Book].[title_id], 
    [Book].[author_id], 
    [Book].[genre_id], 
    [Book].[type],
    CONVERT(VARCHAR(20), [Book].[check_date], 20)
    CONVERT(VARCHAR(20), [Book].[edit_date], 20)
FROM [books] AS [Book] 
WHERE EXISTS 
(
    SELECT * FROM 
    (
        SELECT TOP 1 * 
        FROM [books] AS [bk_inner] 
        WHERE 
            [bk_inner].[title_id] = [Book].[title_id] 
            AND 
            [bk_inner].[type] <> 'paperback' 
    ) AS [bk_outer] 
    WHERE bk_outer.[title_id] = Book.[title_id] 
) 
ORDER BY [Book].[id] desc

We have the final full query. Now how do we get that? First, we need to invoke the getDataSource() method.

class Book extends AppModel {
    . . . 
    function getHardbackBooks(){
        $dbo = $this->getDataSource();

Next we need to use the buildStatement() to build each statement. Since CakePHP will build a sub query with this, we have to do this twice: once for the inner query, and once for the outer query. The “table” for subquery2 will actually be subquery1, so we need to add that as a “table” in the array.

$subquery1 = $dbo->buildStatement(
	array(
		'fields' => array('TOP 1 *'),
        'table' => $dbo->fullTableName($this),
        'alias' => 'bk_inner',
        'limit' => null,
        'offset' => null,
        'joins' => array(),
        'conditions' => 'bk_inner.title_id = Book.title_id AND bk_inner.type <> \'paperback\'',
        'order' => null,
        'group' => null
	
	),
	$this
);

$subQuery2 = $dbo->buildStatement(
    array(
        'fields' => array('*'),
        'table' => '(' . $subquery1 . ')',
        'alias' => 'bk_outer',
        'limit' => null,
        'offset' => null,
        'joins' => array(),
        'conditions' => 'bk_outer.[title_id] = Book.[title_id]',
        'order' => null,
        'group' => null
    ),
    $this
);

Now, we need to make sure we add an EXISTS:

$subQuery = ' EXISTS (' . $subQuery2 . ') ';
return $subQuery;

Return the data from the model to the controller. In the controller function we need to add a new condition to the paginate. In the conditions, we do not need to use a paired item value to set it, we can use the straight SQL returned from the model.

class BooksController extends AppController {
    var $paginate = array(
        'order'        => array('Book.id' => 'desc'),
        'fields'    => array('Book.id', 'Book.title_id', 'Book.author_id', 'Book.genre_id', 'Book.type', 'Book.check_date', 'Book.edit_date'),
        'limit'        => 15,
    );
    . . . 
    function index(){
        $data = $this->Book->getHardbackBooks(); 
        // Set to the paginate object conditions
        $this->paginate['conditions'] = array($data);
        $this->set('hardback_books', $this->paginate());
    }
}

And it returns the items based on the paginate parameters, ready to use in the view. It provides a DISTINCT list. And yes, I know I used more than 400 words in this one. It was closer to 500 without the code. Oh well, maybe tomorrow will be shorter.

You Did What?

Today’s topic is kind of short, but a very important one. If you are not living under a rock, then you know about Sony’s problem with their Playstation online services getting hacked and being down for some time. A new concern now is that this has exposed the credit card numbers of the membership. Something that can definitely cost some good will and trust. However, Sony is a major corporation, and can recover from this. Can your business, if something like this were to happen?

I still see multiple instances of applications (And not just PHP applications) where carelessness has overtaken common sense. The web is no longer just a set of reading materials. It is now more than that with interactive applications and a flow of data that travels in all directions. So why is it that a huge problem is a lack of security for this interaction? The biggest thing I still see is with forms. Multiple sites ask to sign up for something, like a list, and email notification, account to get in to the site, etc. And one of the most powerful things today is information. And this does not mean just credit cards and government identification numbers. These can include names, emails, addresses, cities, passwords, secret phrases for confirmation, etc. Harvesting this can lead to identity fraud, selling to spam lists, etc. Secure your forms! It does not take much time, and can pay off, especially for the small businesses who will not have the money or name recognition of the larger corporations.

Easier said than done, I suspect some are saying. Well, yes and no. This should not be an after thought it should be first. In the PHP language, functions exist to help in this. Some ideas for securing forms: mysql_real_escape_string, pg_escape_string. In fact, if you are using PHP, then make sure to understand the different options available for your database.

That is not all though. You should also use a parametrized approach for inserts and updates. A quick example of this:

$sql = "UPDATE sometable SET somefield = ? WHERE value=?";
$parameters = array($_POST['data1'], $_POST['data2']); 
$dbo->query($sql, $parameters);

Now that was not too hard was it? However, security is not something to pass over. You should understand what data you are collecting, and validate the data, and then securely save the data. Validating it can be as easy as making sure it is an integer value, email, certain number of characters. Items like that can go a long way to verify what you are getting is what you need, and will not harm your application. For example, if a form had a field for first name, last name, country, email address, you can safely validate those fields. First name, last name should only be characters. Those fields should not have special characters, numbers, etc. Email address should be validated against a regular formed email address. You can even go one step further to verify it is a valid email address and exists somewhere out in the cloud.

Big lesson though, secure the data. Secure your application. Do not let a shortcut become your Sony Playstation meltdown.

Have Your Cake and Eat It Too

CakePHP is one of those frameworks where it is easy to set up and get an application running in a minimal amount of time. It provides different securities, helps, and functions in the framework so that your application can run smoothly and be safe. As with all applications, the level of security and functionality depends on the developer, not the code, not the language, not the database. An application is only as secure, functional and reliable as the person/team who is coding it. One of the reasons I do like Cake is that the built in security and helpers offer a developer a great way to secure data, validate it, and display it. And that can also be one of the more trickier parts of getting the application to work correctly, finding the data to do something with it.

CakePHP
CakePHP

CakePHP provides some functionality for finding the data, this is done using the “find” method. You can read more about this at the Cookbook. Using this, one can grab data from many different tables if needed, or just one table. For this exercise, I am going to use dummy data to show how to find data, using simplistic finds, and using joins, and sub-queries. So first lets examine the data tables. Not all of these are going to be connected. This is a very simplistic, quickly drawn up solution to a lending library

Sample Tables
Sample Tables

Continue reading Have Your Cake and Eat It Too

Data Model Relationships – CakePHP’s HABTM

For today, lets dive back into some code, well data modeling at least. When you set up an application that connects to a database, you need to understand the data that will be working in the application. This is the data that will be edited, added, read and even scrutinized int he application. When looking at the application data, one could easily put all data in a table and make it as flat as possible. We could normalize it until the cows come home as well. What is the best choice? My vote is always plan for what is best for the application, and the future of the application. When it comes to data, a more normalized data layout is always going to provide better performance and better ability to scale in the future. In our little example application, we are going to model the data for an online movie rental inventory. We will take an example film: Gran Torino to help the example model.

The data we need for this application includes some basic information: movie title, genre(s), stars, directors, writers, story information, rating, release year, rent price. We can include a lot more data if we really needed to, but for the purpose of this, we will keep it a little simple. A possible way of modeling this data is to create a table that stores all of this information, and have one table in the database. But now when we need to add something else, we have to add columns to the table. For example, in a few months the company decided to add related titles, sequels and sets, etc. It would require a refactor of the data in order to handle this, as well as refactor of the code. So lets split this out.

In the image below, I divided the content based on a few things: Title data, Talent Data, Genre Data, Rating Data

Starting the data model design
Starting the data model design

I now have the four main tables, but we need to figure out how these are related. First lets tackle the Rating Data, as that will be a simple design. I am linking to the IMDB so you can look at the data. The ratings available in the United States, at least the ones we will include, are: G (General Audiences), PG (Parental Guidance suggested), PG-13 (Parents strongly cautioned) and R (Restricted, no one under 17 allowed without a parent, or as I call it, PG-17). So each rating will be housed in this table. We will need an identifier, the rating, the explanation, and some data to track creation and modification. We do not need those last two, but it is just good practice to include those if there is ever going to modification on data. Using our example film, it is rated “R”. And since any film title object (Titles) will ever only have one rating (for the sake of this example) there is an easy relation of a hasOne relation to the Ratings table. We need to add a foreign key to the Titles table and connect these.

hasOne Relation to Ratings
Title hasOne Rating

Easy to connect those. Now, we need to tackle the Genres. This is a little more complicated, but we can get through this. The Genres table will house the genres we need to display. This list can be as big or small as needed. Our example movie is in the “Drama” genre according to IMDB. However, in our application, the business has decided the movie is classified as Drama and Action. So now a title is going to have many genres. And a genre can belong to many titles. The “Drama” genre may belong to multiple titles. So we can not just add a new column to the Titles table, as that will not satisfy the requirements. We need to add a connecting table, and according to the naming convention of CakePHP, the name of the connector table is the alphabetical order of the two tables it is connecting.

So we need to add a table titled “Genre_Titles”. It will have an ID, and foreign keys to both tables.

HABTM Genres
Connecting the Genre and Title tables

Now we are almost done with the HABTM set up. We made it through one of them, and that was a good thing. See it was not so difficult. Now, we need to finish this up, and connect the talent table to the title. Talent can be anything. Since the company wants to display the stars of the show, the directors and writers, we need to be able to connect these. And again, this will require a a HABTM relationship. An actor can be in many titles, just like Clint Eastwood, as he was not just in Gran Torino. So he may be listed in many titles. And, with this movie, he not only stars in it, he directed it. So now we not only need to match up this talent, we have to identify it correctly. So this adds a little complexity to this, but we can do this.

As you know from the previous example, we need to create a connecting table. The name would be “Talent_Titles”. But that still will not solve the issue of identifying Clint Eastwood as an actor and director in the title. We can add a new table “Talent_Types”. This will be a “lookup table” that houses Star, Director, Writer as values. We can then connect that to the connecting table. This relationship will be a hasMany to Talent_Titles, as a star may have many entries in the Talent_Title table.

HABTM Talent and Title
HABTM Talent and Title

And that is the HABTM design. Using the Bake method, you can now bake this up, and set up the model. The thing to remember about the HABTM, it is not something to fear. Usually, if a connecting table is needed, you have a HABTM design. Remember to think in human terms when examining the data model. What does this belong to, what does it have. In this, the Title will have many actors, directors, etc. And the actors, directors, etc will belong to many titles.

Thinking of the site

So as I am sitting here trying to get caught up on the StephenHird site, I am thinking a little more about it. It seems like it is just a huge issue right now and am not able to complete what I really want to complete on the site. So I may make an executive decision to abandon the code (not destroy it, but abandon it and store it safely if I want to use it again), start over from scratch, again using CakePHP, and the Facebook Graph API. This would be a little less intense, and would contain an easier example of how to get some data out there, and still include the Graph API in order to show examples of how to do this.

I still think the idea of the resume integrated with the API is a good one to show an example, but I am finding myself with less time to do a full blown app for it. And I figure, if I do a small example, that should be enough, and there is always the documentation available at Facebook for this. So if you have been following the Graph API integration, it still will happen, just in a different form.

Getting into the code

So it has been a couple of weeks since I have posted. But now we need to set up some base code before we can go forward with the details and then adding in the Facebook Graph API. In the last post, the Data model was set up. We have skills and certifications as standalone tables. Skills with the levels and areas tables connected together. We also have the main glue of resumes, connected to covers and tasks which itself is connected to jobs. A lot of tables to create the resume section, but will keep some of this information all together. We need to create some code so that we can get all this information.

If you Baked each object, and used the Bake methods to create the model, and associations, as well as the controllers an views, you will have some code ready to use and ready to go. After you have Baked these items, the sample code that is created is ok to begin with. However, we want to take advantage of a very important technique, and the is the centralization of code, and prevent code duplication. There is one other thing that gets to me, and this is more of an OCD thing for me in code, and that is the way that Cake does the edit check in the controller. In the base created code, it creates a section of code that checks for an ID. If it is not passed, then it redirects the page elsewhere. Like so:

function edit($id = null) {
    if (!$id && empty($this->data)) {
        $this->Session->setFlash(__('Invalid resume', true));
        $this->redirect(array('action' => 'index'));
    }
    . . . . 
}

So in this code segment, if one gets to the edit form, and an ID is passed, and the form is not filled in, then it will display the actual form. And if the ID is not passed in, then it redirects to the Index page. For examples, if the site name was test.com:
www.test.com/resume/edit/2 – will result in the form being shown
www.test.com/resume/edit – will result in a redirect and the error message

Now here is where my OCD kicks in a little. . . .
Continue reading Getting into the code