Historical context and evolution of CGI.

When you click on any of the links on the webpage, your browser contacts the HTTP web server and demands the URL that is, the filename, and the web server passes the URL and looks for the filename, if it finds that file, it will send it back to the browser otherwise sends an indicating message that you have requested a wrong file. Then, the web browser takes a response from the web server and displays either the received file or the error message. However, it is possible to set the HTTP server so that whenever a file in a certain directory is requested, that file is not sent back. Instead, it is executed as a program. And, whenever that program shows any output, it’s sent back to the browser to display. So, this function is called Common Gateway Interface or CGI, and the program scripts are called CGI Scripts. These CGI scripts can be a Python script, PERL script, Shell script, C or C++ program, etc.

What is CGI?

CGI or Common Gateway Interface, is the industry-accepted acronym for a set method that is used to define how information is exchanged between the web server and a custom script. It is a protocol that defines how web servers and external programs or scripts can communicate to generate dynamic content on the World Wide Web.

Why CGI?

A document collection directory, or a bundle of files that may be transmitted to Web browsers linked to the server, is typically present on a Web server. The web server will respond to a request for http://www.example.com/index.html by sending the browser a copy of the file /usr/local/apache/htdocs/index.html (if it exists), for instance, if a web server has the fully qualified domain name www.example.com and its document collection is stored at /usr/local/apache/htdocs/ in the local file system (its document root). The server software may delay requests for dynamically created pages to different programs and then relay the results to the requesting client (often a Web browser that renders the page to the user).

Such applications typically need some more information, such as query strings or s, to be included with the request. On the other hand, when responding to a request, the script is expected to provide all the data needed by HTTP, such as the HTTP status of the request, the document content (if available), the document type (such as HTML, PDF, or plain text), etc.

Initially, there were no standard methods for data exchange between a browser, the HTTP server it was connecting with, and the scripts on the server that were anticipated to process the data, finally sending a result to the browser. As a result, there were incompatibilities between various HTTP server variations that made script porting difficult. The specification of how data sharing was to be done as a result of this problem’s recognition led to the creation of CGI. CGI scripts are web page-generating applications that are called by server software that complies with the CGI specification, even if they may actually have been developed in a non-scripting language, such as C. All well-known HTTP server packages, including Apache, Microsoft IIS, and (with an extension) node.js-based servers, swiftly embraced and still support the CGI protocol. Processing forms was one of the earliest uses of CGI scripts. In the early days of HTML, forms often contained a button labeled “submit” and a “action” attribute. Pushing the submit button would send the form’s data as a query string to the server at the URI specified in the “action” property. An HTML page would be produced if the “action” specifies a CGI script, which would then be executed.

Features of CGI

As we came across different aspects of CGI above, let us know more about some of its features which makes it useful and essential for us. They are given below:

1. Dynamic Content Generation: On web servers, CGI enables the creation of dynamic content. Websites are now able to show dynamic data that can vary in response to user input, database queries, or other real-time events because it makes it possible to run external programs or scripts in response to HTTP requests.
2. Flexibility: CGI offers developers flexibility while putting web applications into action. It makes it possible to construct a wide variety of online apps and services by allowing for the specific processing of requests and responses.
3. Extensibility: CGI scripts can be set up to run on web servers in response to particular URL paths or file extensions. Developers can decide when and where to employ CGI scripts due to extensibility.
4. Compatibility: CGI is still compatible with the majority of web servers and hosting setups, despite its age and the development of new web technologies. Due to this backward compatibility, many old CGI programs can still be used.
5. Stateless: Because CGI is stateless, each HTTP request is handled separately. Although its simplicity makes it simple to use, it can also cause performance problems when managing a lot of queries.
6. HTTP Data Access: CGI scripts can access and process data from HTTP requests, including form data submitted by users through web forms, query parameters, and cookies. This enables the creation of interactive web applications.
7. Standardized Protocol: CGI stipulates a standard protocol for dialogue between the web server and external applications. Regardless of the underlying technology, this protocol makes sure that web servers can interact with CGI scripts in a uniform way.
8. Language Independence: Because CGI is language-independent, it can run scripts or programs created in a variety of programming languages. Although it can be used with other languages as well, the earliest CGI scripts were frequently written in languages like Perl, C, or shell programming.
9. Server-Side Processing: Business logic and data processing are both possible through the server-side execution of CGI scripts. Tasks like user authentication, database interactions, and content creation depend on this.
10. Testing and Debugging: Testing and debugging CGI scripts locally before deployment makes the development process simpler and lowers the risk of production-level mistakes.

CGI Architeture

When a request is made from the Web Server to the Client Server, the Common Gateway Interface (CGI) uses external files or programs to handle such requests and these programs or files can be written in any programming language. But, the main goal of these programs is to retrieve data from the databases, and the data retrieved from the databases is converted into HTML format and sent to the Web Server.

Historical Context of CGI

In 1993, the standard for invoking command line executables was written by the National Center for Supercomputing Applications (NCSA) team and posted to the www-talk mailing list. It was embraced by the other Web server creators, and ever since then, Web servers have used it as a standard. A work group headed by Ken Coar was established in November 1997 to formalize the NCSA definition of CGI. This effort led to the creation of RFC 3875, which described CGI Version 1.1. Specifically mentioned in the RFC are the following contributors:

• Rob McCool, the author of the NCSA HTTPd Web server
• John Franks, the author of the GN Web server
• Ari Luotonen, the developer of the CERN httpd Web server
• Tony Sanders, the author of the Plexus Web server
• George Phillips, the Web server maintainer at the University of British Columbia

In ancient times, C programming language was popularly used for creating CGI apps. RFC 3875 “The Common Gateway Interface (CGI)” has partially defined CGI using C, in saying that environment variables “are accessed by the C library routine getenv() or variable environ”.

When webmasters first started using the Web and wished to link existing information systems, such as databases, to their Web servers, they gave the technology the term CGI. The server that served as a common “gateway” between the Web server and the older information system, ran the CGI application.

Evolution of CGI

Since it was first introduced in the early 1990s, the Common Gateway Interface (CGI) has seen tremendous development. The evolving demands of web development and technological advancements have influenced its progress. An overview of its evolution is shown below:

1. Early Year and Origins (1990s)

CGI was developed in response to the early World Wide Web’s demand for dynamic content. The CGI protocol was first established by the NCSA at the University of Illinois in 1993.

Initially, Perl and C were the primary languages used to create CGI scripts. These scripts handled database queries, processed form data, and produced dynamic HTML pages.

2. Performace Obstacles

Due to the fact that each request resulted in a new process, early CGI experienced performance issues. Performance issues resulted from this inefficiency, particularly when there is a lot of web traffic.

3. Server-side Scripting Language (Late 1990s – Early 2000s)

Server-side scripting languages like PHP, Python, Ruby, and Java have become more popular as a solution to performance problems. Compared to CGI, these languages provided more effective techniques to create dynamic web applications.

4. Web Frameworks and Abstraction (2000s – Present)

By offering formal methods for creating dynamic applications, web application frameworks (such as Ruby on Rails, Django, and CGI::Application for Perl) were introduced. The management of HTTP requests and responses was abstracted.

5. FastCGI AND MOD_PERL

Technologies like FastCGI and Mod_Perl were introduced to boost CGI’s performance. FastCGI reduced the overhead of creating processes by enabling CGI programs to run as persistent processes. Similar performance improvements for Perl programs were offered by Mod_Perl.

6. Transition to Web Service and APIs (2000s – Present)

Digital services and APIs have become increasingly prevalent in today’s digital environment. Developers frequently design APIs that feed data to front-end applications rather than creating whole web pages using CGI or server-side scripting.

7. Serverless Computing and Cloud Service (2010s – Present)

Web development has been further changed with the emergence of serverless computing platforms and cloud services (such as AWS Lambda and Azure Functions). As a result of these platforms’ abstraction of server administration, programmers can create and deploy compact, event-driven routines that reply to HTTP requests.

8. Legacy and Influence

Despite its diminished popularity, CGI’s influence can still be seen in the way that modern web programming still separates display from functionality and uses request-response paradigms.

Advantage and Diadvantage of CGI

Advantage of CGI

Some of the advantages of Common Gateway Interface are:

1. CGI scripts are portable in nature.
2. They are language independent, as we can write them in any language, instead of being stuck to any specific language.
3. CGI can perform both easy and complex tasks efficiently, which makes it scalable as well.
4. It also helps in dynamic communication as well.

Diadvantage of CGI

Along with advantages, CGI or Common Gateway Interface also has some of its disadvantages. Those are:

1. The CGI scripts may create a lot of traffic since there are a lot of requests from the client-server.
2. Sometimes, it maybe very complex to design and program the CGI scripts.
3. CGI programs maybe a little vulnerable regarding the security considering how freely they are available.

Steps to Run and Configure the CGI Script

Step 1: ENABLE CGI SUPPORT IN APACHE

Before proceeding with the CGI programming, ensure that there is a web server setup on your system. Here, XAMPP server is used, as it will help in setting up the server Apache 2.0 on the system.

Also, before running or configuring the CGI scripts in Apache, you need to first check for enabling the CGI support in Apache. You can do so by accessing the httpd.conf file from the Config option.

When you open the httpd.conf file, locate to the below mentioned line, and remove the ‘#’ sign from its beginning (if it is not removed).

LoadModule cgi_module modules/mod_cgi.so

Step 2: CREATE A CGI SCRIPT

Once the CGI support is enabled, you can start creating your CGI scripts, using any of the Python, Perl or Shell Script. You can use any of the text editors to create the scripts and save it with .cgi extension.

Here, is a small CGI program on Hello_mycgi.py

#!C:\Python311\python.exe
print("Content-type:text/html\r\n\r\n")
print('<html>')
print('<head>')
print('<title>CGI Hello World</title>')
print('</head>')
print('<body>')
print('<h1>Hello, World!</h1>')
print('<p>This is a simple CGI program.</p>')
print('</body>')
print('</html>')

Note − First line in the script must be the path to Python executable. It appears as a comment in Python program, but it is called shebang line. If you will not put this line in your script then, you may encounter multiple errors and difficulties while running the program.

Step 3: CONFIGURE APACHE TO RECOGNIZE THE CGI SCRIPT

Once you are done with creating the CGI script, next you must configure Apache to recognize the script as the CGI script. For this, you need to do make the specific configurations in the Apache httpd.conf file to make sure that the web server is configured with all types of CGI program handlers. First of all, in the “AddHandler” column, make sure there has to be a .cgi and a .py extension, as shown below

AddHandler cgi-script .cgi .pl .asp .py

This will allow the Apache server to treat .py file as CGI script. Also, check for the “ExecCGI” in the “Options” directive, as shown below

This will allow to use CGI scripts outside of ScriptAliased directories.

All the CGI programs which are to be executed by the HTTP server are kept in a pre-configured directory, called as CGI Directory or C:/xampp/cgi-bin. Although, by convention, CGI files have .cgi as extension, but it can have Python extension .py as well.

By default, the Linux server is set up to only execute the scripts found in the cgi-bin directory under C:/xampp/cgi-bin. The following lines in the httpd.conf file should be commented if you want to choose any alternative directory to run your CGI scripts in. Save the configuration file and restart the Apache web server for the changes to take effect.

Step 4: TEST THE CGI SCRIPT

Now, that you are done with all the configurations, finally its time to test the CGI script. To do so, simply open your web browser and type the address of your script in the URL.

This is the output which you will receive on the STDOUT file, i.e., screen. The line Content-type:text/html\r\n\r\n will be sent back to the browser to specify the content type to be displayed on the browser screen.