Archive for the ‘open source’ Category

Column-oriented MySQL for VLDB

InfiniDB – Open Source BI/Analytic Database

Recently ran across some blog posts about InfiniDB, a MySQL based DW and BI analytic database from a company called Calpont. I *think* InfiniDB is their only product. I could be wrong about that, though.
There are plenty of things to like about InfiniDB – Multi-threaded and designed for multi-cpu/cores, ACID compliant, recoverable, supports SQL standards and online DDL, MVCC, dynamic data compression, and FREE! What attracted me first though was the open source implementation of the columnar storage. That’s the current biggie on VLDB, think vertica or Oracle’s ExaData.

Click to continue reading “Column-oriented MySQL for VLDB”

Read the rest of this entry »

Cool Tool Of The Week – Open Modelsphere: Open Source Data Modeling

From The Database Geek

I like playing with new tools and software so I thought I would start a new weekly series, Cool Tool of the Week. I will just cover a tool that I have used or discovered that I think is cool. I’m not necessarily endoring it, just giving it a little publicity so that others will know about it and can give it a try.

This week is the open source data modeling tool Open ModelSphere. This Open Source project is available for windows with a GPL license. It was originally known as SilverRun. I used this windows-based CASE tool when I was at Yale.

Click to continue reading “Cool Tool Of The Week – Open Modelsphere: Open Source Data Modeling”

Read the rest of this entry »

PL/JSON v0.6.2 Released

It was pointed out to me that I had a few errors in the previous release.  Those have been fixed and the install should go a lot smoother now.  No additional functionality here, just bug fixes.  If you tweaked the previous release to install it, you don’t need this one.

As always, you can download the new code directly from the PL/JSON SourceForge page.

Change Log:

Version: 0.6.2
Fixed exit commands in various code files
Removed the “CREATE OR REPLACE” on the string_handler.printf function

Version: 0.6.1
Added missing string_handler package

Version: 0.6
Added support for creating JSON from text input

Version 0.5
Initial release
Includes a JSON data type
Can create tables and columns of JSON
Supports API creation of JSON data type

LewisC

Read the rest of this entry »

Oracle INFORMATION_SCHEMA

From the Database Geek.

Part of the ANSI SQL standard calls for an INFORMATION_SCHEMA. This schema contains a standardized data dictionary that is (or is supposed to be) common across various databases. Most database vendors offer a native data dictionary and a sub-set INFORMATION_SCHEMA (called info schema from this point on). Oracle is the only major database vendor (that I know of) that doesn’t even offer a subset of the info schema.

Oracle’s data dictionary (especially if you include the V$ views in that) is the most robust of all the databases I have used.

Click to continue reading “Oracle INFORMATION_SCHEMA”

Read the rest of this entry »

PL/JSON v0.6 Released to SourceForge

From the Database Geek.

PL/JSON v0.6.1 has now been released to Sourceforge.net. The reason for the .1 is that I forgot a dependency on the 0.6 release.

Anyway, PL/JSON can now create JSON via the API or through parsing a text string. The parser is implemented as a stand along package. That means you can write your own parser (or use an external procedure) if you don’t like mine. This will make it very extensible. I also plan to add PL/SQL callbacks to the parser at some point for custom processing.

Example of parsing:

SET SERVEROUTPUT ON

DECLARE
  v_json json;
  v_json2 VARCHAR2(32000) :=
   '{
      "abc": "dpkxvdvvcxz\"vxasasa   ",
      "def": 12345,
      "ghi":{"isit":true, "nope":false,"denada":true },
      "jkl": [1234, 45678.99, 121211, 21323232, 00000]
   }';
BEGIN

  v_json := json(v_json2);
  v_json.print;
END;
/

{
"abc":"dpkxvdvvcxz\"vxasasa   ",
"def":12345,
"ghi":{
"isit":true,
"nope":false,
"denada":true}
,
"jkl":[1234,45678.99,121211,21323232,0]}

Output from PL/JSON checks out on JSONLint.

Click to continue reading “PL/JSON v0.6 Released to SourceForge”

Read the rest of this entry »

ORA_Tweet Now Does Short URLs

Paul Gallagher, of the Tardate blog, emailed me this morning (or I guess it was last night) and sent me some code to enhace ORA_Tweet.  He wrote a package to call out to an is.gd API to convert a long URL to a short URL.  The really cool thing is that he used a regular expression to find matches INSIDE the comment body.  That means that ANY URLs will be replaced with a short URL, you don’t have to call the API individually.

His API is easily called and can be used for projects other than just ORA_Tweet.  Check out his write up on the SHORTURL package:  Tweeting from your database with short urls

I updated the ora_tweet code so you can download the entire set of code if you would like.

Click to continue reading “ORA_Tweet Now Does Short URLs”

Read the rest of this entry »

Catching Up – Email, PL/JSON, Oracle Open Source, ORA_Tweet on oracle.com

Hi all.

This post is just a catchup of what I have been doing recently.

I cleaned out my email inbox. I had 5000 unread emails. I have a problem that if I get an email that is not spam but is not critical, I put it aside “to read later”. Sometimes, I actually do go back and read them. More often, I never see it again. I would *LIKE* to respond to every email and blog post that I get. Unfortunately, I just don’t always have time. By the time I do have time, I have plenty of newer posts and emails.

Click to continue reading “Catching Up – Email, PL/JSON, Oracle Open Source, ORA_Tweet on oracle.com”

Read the rest of this entry »

JSON In (and out) of Oracle – JSON Data Type

From the database geek.

Update March 29, 2009:  PL/JSON is now available as an open source project (MIT License).  Read the FAQ at http://oracleoss.com

Do you use JSON? Do you use Oracle? How would you like to use JSON in Oracle? How would you like to store it, generate it and read it? I am creating a new JSON data type that will work like XMLType.

I am assuming you are familiar with Oracle (or you probably wouldn’t be reading this blog). If you are not familiar with JSON it is JavaScript Object Notation, a lightweight data-interchange specification. Think of it as a replacement to XML.

Click to continue reading “JSON In (and out) of Oracle – JSON Data Type”

Read the rest of this entry »

Ora_Tweet – Tweet From Oracle, A PL/SQL Twitter API

Get the latest source from the ORA Tweet project page at sourceforge.

I had some free time this last Saturday night (after the family was all in bed) and I wanted to do a little mini-project. I don’t get to do enough PL/SQL anymore so I like to look for utilities and smaller fun things to code. While I was thinking about it, I was also following along on twitter to the posts that the people I follow had recently made.

I decided to put together a twitter procedure that would let me make posts from the database.

Click to continue reading “Ora_Tweet – Tweet From Oracle, A PL/SQL Twitter API”

Read the rest of this entry »

SQL Analytics, With Clause and Column Level Grants

From the database geek.

Postgres 8.4 is just around the corner. I haven’t had much time lately to keep up with Postgres. I’ve had too much to do in the Oracle world (new job, writing, etc). However, I do try to keep up with a few email groups just to keep touch with what’s coming. Here are three things in Postgres 8.4 that I think will be interesting to a lot of people. Two of them bring Postgres that much closer to Oracle and one of them is useful (at times).

SQL Analytics

SQL Analytic Functions, also called Window Functions, allow you to execute functions across multiple sets of data within a single query.

For example, you can query the max value (or sum, average, whatever) of a field for multiple sort orders in a single query.

Click to continue reading “SQL Analytics, With Clause and Column Level Grants”

Read the rest of this entry »

Percona Offers InnoDB Replacement

Open source the way it ought to be. Today, Percona announced a replacement for InnoDB that improves performance and fixes bugs. The new engine is called XtraDB.

According to Vadim at Percona:

It’s 100% backwards-compatible with standard InnoDB, so you can use it as a drop-in replacement in your current environment. It is designed to scale better on modern hardware, and includes a variety of other features useful in high performance environments.

The release is pure GPL (v2) and commercial support is available from Percona. If percona keeps this up, they just might become the new MySQL.

The source is available from Launchpad and from Percona.

Click to continue reading “Percona Offers InnoDB Replacement”

Read the rest of this entry »

Starting Survey results – Survery Countries

Well, I have started formatting the results from the survey. This will take a little while as the survey software doesn’t make it easy to download and clean it up (without paying for a subscription). As soon as the data is cleaned up I will post the entire data set and a link for everyone to download.

However, while I work on the data, I will provide some summary results. Here is a list of countries who had respondents. If a country is not listed, it had 0 responses/

Country

Percentage

# Respondents

Argentina

0.31%

1

Australia

3.44%

11

Belgium

1.25%

4

Bosnia and Herzegovina

0.31%

1

Brazil

5.63%

18

Bulgaria

1.25%

4

Canada

4.69%

15

Colombia

0.63%

2

Croatia

0.31%

1

Czech Republic

0.94%

3

Dominican Republic

0.31%

1

Ecuador

0.31%

1

Estonia

0.31%

1

Finland

0.31%

1

France

4.38%

14

Germany

2.50%

8

Greece

0.63%

2

Guatemala

0.31%

1

Hong Kong

0.31%

1

Hungary

0.31%

1

India

3.13%

10

Indonesia

0.31%

1

Israel

0.31%

1

Italy

1.88%

6

Latvia

0.63%

2

Malaysia

0.31%

1

Mali

0.31%

1

Mexico

3.75%

12

Netherlands

1.88%

6

Norway

0.31%

1

Paraguay

0.31%

1

Peru

0.63%

2

Philippines

0.63%

2

Poland

1.88%

6

Portugal

0.31%

1

Russian Federation

1.56%

5

Serbia

0.63%

2

Singapore

0.31%

1

Slovenia

0.94%

3

South Africa

0.94%

3

Spain

2.81%

9

Sweden

0.94%

3

Switzerland

1.25%

4

Thailand

0.31%

1

Turkey

1.25%

4

Ukraine

0.31%

1

United Kingdom

6.88%

22

United States

35.00%

112

Uruguay

0.31%

1

Other

1.56%

5

Check out a summary of Primary Databases and a few questions on open source software usage.

LewisC

Over 200 Responses in Less than 2 Weeks

Less than two weeks ago, I posted my Database Survey. As of just a few minutes ago, I have had 215 responses. That’s pretty awesome. I’d like to get at least twice that though.

I haven’t looked deeply at it yet to see if there are any trends. I think it will be best to wait until the survey is closed. I did look at some of the responses, kind of as a quality check. Looks like MySQL is fairly well represented. I didn’t see any DB2 responses (for primary database).

Click to continue reading “Over 200 Responses in Less than 2 Weeks”

Read the rest of this entry »

Infoworld Picks MySQL as Best Database

Infoworld published the 2008 Bossies, Best Of Open Source Software. There are 8 categories and none of them are database:

  • Collaboration
  • Developer tools
  • Enterprise applications
  • Networking
  • Platforms and middleware
  • Productivity applications
  • Security
  • Storage

I had to look through several of them before I found the database category under Platforms and middleware. Slide 4 is the magic slide:

It says:

Database

While SQLite3 is extremely convenient for development and testing databases, and PostgreSQL has powerful Generalized Search Tree indexes and is very close to being enterprise-ready, is the choice for many Web sites thanks to its excellent read performance, transparent support for large text and binary objects, and incredibly easy administration.

Click to continue reading “Infoworld Picks MySQL as Best Database”

Read the rest of this entry »

High Performance MySQL: Review

High Performance MySQL, Second Edition

Optimization, Backups, Replication, and More

By Baron Schwartz , Peter Zaitsev , Vadim Tkachenko , Jeremy Zawodny , Arjen Lentz , Derek J. Balling

Pages: 708

ISBN 10: 0-596-10171-6 | ISBN 13: 9780596101718

When I first read about this book, I figured many sections would be over my head. I was pleasantly surprised when I started reading it.

Click to continue reading “High Performance MySQL: Review”

Read the rest of this entry »

LinkedIn Buys Into MySQL

Hot on the heels of news that SquareSpace is using Oracle, comes news that LinkedIn is going whole hog with MySQL.

Actually, you could say that LinkedIn is buying into Sun. They are buying the MySQL Enterprise subscription and they’ll be running MySQL on Sparc servers and Solaris 10. They’ve signed up for Sun Professional Services, MySQL Professional Services, and Solaris Everywhere. I guess you could say that signed up for the full monty. ;-) Pun intended.

Helping LinkedIn to scale their Web systems demonstrates the strength of combining the Sun and MySQL teams, said Zack Urlocker, vice-president of products, database group, Sun Microsystems.

Click to continue reading “LinkedIn Buys Into MySQL”

Read the rest of this entry »

Please take a 25 question survey

I recently posted about the results of the EDB Open Source Survey. It doesn’t look like the raw data is going to be released. I was also reading about an O’reilly survey that costs $350 to see. Not that I am opposed to them making money, but I wanted to see the results and I think there are a lot people who feel the same. I have no idea if my survey is anything like O’reilly’s.

Anyway, I have created a new survey. It has about 10 demographics questions asking who you are, where you are, what kind of role you have, etc.

Click to continue reading “Please take a 25 question survey”

Read the rest of this entry »

Help Me Out – Take a survey

Hi. I’ve created a 25 question survey on databases and open source. There are about 10 demographic type questions (who are you, where are you, etc) and then a bunch of questions on databases and open source.

The results of this survey, and by results I mean all of the raw data, will be released to the public at the completion of the survey.

I ask for an email ID so that I can send you the results. This is totally optional and you will still be able to get the results should you chose not enter one.

Click to continue reading “Help Me Out – Take a survey”

Read the rest of this entry »

OSCON 2008 Popularity Contest

I didn’t get a chance to go to OSCON 2008. Bummer. But I can live vicariously through google. So, along with all of the announcements you’ve heard from OSCON, I know present the OSCON 2008 – Google popularity contest. This is a completely unscientific survey of google hits. I was searching blogs and news. I started with just news but the blogs hits really upped the numbers.

To run these searches, I use “oscon 2008″ and the search term, for example:

“oscon 2008″ mysql

In the case of open source, I also quoted “open source”.

I’m using google’s about number. I didn’t sit and count each hit.

Click to continue reading “OSCON 2008 Popularity Contest”

Read the rest of this entry »

Update on EDB Open Source Database Survey

Yesterday I posted about the results of the EnterpriseDB open source survey. In that post, I said:

I’d like to see the survey again and compare the results to the survey itself.

I discovered that, as of right now, the survey is still online.

Still no information as to when, or if, the entire survey results will be released. That’s what I am most interested in.

LewisC


Technorati : , , ,

Read the rest of this entry »

Is Drizzle good for MySQL?

Have you heard of Drizzle? It was announced at OSCON yesterday and is all over the blogosphere. From the Drizzle FAQ:

* So what are the differences between is and MySQL?

No modes, views, triggers, prepared statements, stored procedures, query cache, data conversion inserts, ACL. Fewer data types. Less engines, less code. Assume the primary engine is transactional.

Also from the FAQ is that, right now at least, there is no intention to make this run natively on windows and they make the point:

* “This is not a SQL compliant relational…”

Very true, and we do not aim to be that.

It is a fork of MySQL that takes it backward to pre-5.0 in features but hopefully greatly reduces the bugs and instabilities.

Click to continue reading “Is Drizzle good for MySQL?”

Read the rest of this entry »

Results of EnterpriseDB Open Source Database Survey

EnterpriseDB announced the results of the survey they did a few months ago at OSCON. Now, take the results with a grain of salt as it was done by EnterpriseDB. EnterpriseDB is based on Postgres so there is a vested interest in making Postgres sound good. Results can be skewed depending on how the survey is worded, what options are available as answers and who the respondents are.

The results summary is available for free.

Some key facts:

500 respondents. The download page says “500 corporate IT leaders”. Or maybe, 500 open source developers. ;-)

Only 9% of respondents indicated that they preferred commercial solutions over open source solutions.

Click to continue reading “Results of EnterpriseDB Open Source Database Survey”

Read the rest of this entry »

MySQL vs Postgres, Again – Is Postgres Better?

I was browsing the web on this lazy Sunday afternoon and ran across a good article on the Rarest Words blog. The author was trying to get Django installed and running with Postgres. From the author’s own admissions, he is not a Postgres fanatic.

Well, this and last year I hear everywhere that PostgreSQL is the way to go and that usage of mySQL in 2008 makes people puke… But without any real arguments (besides “Postgres is the way to go”).

After some not so compatible errors with these not so compatible databases, the author did get it working and ran some benchmarks. Postgres did not turn out faster than MySQL.

Click to continue reading “MySQL vs Postgres, Again – Is Postgres Better?”

Read the rest of this entry »

EDB Tip #5: Tablespaces and Tables in Postgres

LewisC’s An Expert’s Guide To Oracle Technology

The datastore in Postgres (which is what Advanced Server is based on) is quite a bit different than the way Oracle stores data. Gone are the usual logical storage components. Data blocks, extents and segments just don’t exist in a Postgres database. Actually, segments do exist when a table gets bigger than 1GB but that’s a story for a different entry.

Postgres stores data as files and directories. Tablespaces are directories and tables (and indexes) are files inside that directory. Postgres ships with two tablespaces installed by default: pg_default and pg_global. The tablespaces cannot be changed or dropped.

Click to continue reading “EDB Tip #5: Tablespaces and Tables in Postgres”

Read the rest of this entry »

Free Database Design Tools

LewisC’s An Expert’s Guide To Oracle Technology

Sun just announced MySQL Workbench, a new database design tool for MySQL developers and DBAs. I’m a data modeling tool junkie. I like to play with any I can get my hands on. I’ve used almost every modeling tool that’s been built. My all time favorite is probably Erwin.

I decided to download MySQL Workbench and give it a try. Since I was playing with it, I figured I should write about it and while I am writing about it, I might as well write about a couple of other tools, that I have personally used, that you might like.

TOAD Data Modeler

The TOAD Data Modeler from Quest used to have a free version.

Click to continue reading “Free Database Design Tools”

Read the rest of this entry »

Wavemaker Provides EnterpriseDB Support

EnterpriseDB News at Blogspot

Chris Keene, the CEO for Wavemaker just blogged that combining Wavemaker and EnterpriseDB just got easier. Wavemaker has a new version that has out of the box support for both Postgres and EnterpriseDB.

I wrote a couple of weeks ago about my first day of Wavemaker training. I still haven’t found time to start day two but now that Wavemaker supports EnterpriseDB, I’ll probably redo day 1 using Advanced Server instead of MySQL. I could have done it before since it already supported Postgres and EnterpriseDB via a manual configuration.

Click to continue reading “Wavemaker Provides EnterpriseDB Support”

Read the rest of this entry »

World’s Largest Database Runs on Postgres?

LewisC’s An Expert’s Guide To Oracle Technology

According to an article at Computerworld, Yahoo is running a 2 PB (not GB, not TB, PB – Petabyte) database that processes 24 billion events a day. Let’s put that in perspective. 24 billion events is 24,000 million events; 24,000,000,000 events. 1 petabyte is 1,000,000,000,000 bytes. Yahoo has two of those. Actually, I should be basing this on 1k which is 1024 but when you’re dealing with petabytes, I don’t think we need to be picky. We’re talking really, really big.

Yahoo uses this database to analyze the browsing habits of it half a billion monthly visitors.

Click to continue reading “World’s Largest Database Runs on Postgres?”

Read the rest of this entry »

Take an Open Source Database Survey

LewisC’s An Expert’s Guide To Oracle Technology

Do you know which open source feature is the most important? Do you know which open source database rocks and which one sucks? Is MySQL better than Postgres? Is Ingres worth considering? How does Firebird compare? Have you used, or have you considered using, an open source database?

Take a survey. It’s only 15 questions so it takes just a few minutes.

I’ll post a link where you can get the results once they have been compiled and prepared.

BTW, this isn’t my survey. I’m just passing on the link.

LewisC

Read the rest of this entry »

Wavemaker Training Day 1

LewisC’s An Expert’s Guide To Oracle Technology

I mentioned the other day that I was playing with a new GUI builder from an EnterpriseDB partner, Wavemaker. Wavemaker, in addition to providing the software, provides some training that is downloadable from the web site.

This training is a two day basics course in powerpoint format. I would guess that this is what they use for instructor led training and they just made this available to the public. Anyway, I decided to see how good the training was which in turn kind of tells me how easy wavemaker is to learn.

Day 1 includes 8 powerpoint presentations:

  • Introduction
  • Wavemaker Architecture
  • Wavemaker Installation
  • Wavemaker Studio Overview
  • Page Designer Basics
  • Wavemaker Data Model
  • Service Components
  • Data Widgets

Seeing as how I was already making some simple pages with the tool, I blew through the intro, the architecture and installation.

Click to continue reading “Wavemaker Training Day 1″

Read the rest of this entry »

EnterpriseDB Open Sources GridSQL

I wrote about GridSQL and EnterpriseDB a while back. EnterpriseDB bought GridSQL (from Mason who now works at EnterpriseDB) and has been working to improve it. Yesterday, they announced that they are open sourcing it.

GridSQL is really for data warehousing but can be applied to some OLTP/reporting combination databases. What it does is take a query and spread it out to run in parallel against a set of cheap PCs. There is a controller machine, called a coordinator, that keeps track of what is running and where.

Because there is a central point of failure (the coordinator), it is not a high availability solution.

Click to continue reading “EnterpriseDB Open Sources GridSQL”

Read the rest of this entry »