Friday, August 31, 2012
Cutts & Mueller On PageRank Penalties & Selling Links
Cutts & Mueller On PageRank Penalties & Selling Links: Yesterday, both Matt Cutts and John Mueller of Google answered the same question on PageRank penalties for selling links without knowing the other answered them...
DIGITAL JUICE
DIGITAL JUICE
Photonic interactions measured at atomic level
Photonic interactions measured at atomic level: By measuring the unique properties of light on the scale of a single atom, researchers believe that they have characterized the limits of metal's ability in devices that enhance light.
DIGITAL JUICE
DIGITAL JUICE
NASA's Jupiter-bound Juno changes its orbit
NASA's Jupiter-bound Juno changes its orbit: Earlier today, navigators and mission controllers for NASA's Juno mission to Jupiter watched their computer screens as their spacecraft successfully performed its first deep-space maneuver. This first firing of Juno's main engine is one of two planned to refine the spacecraft's trajectory, setting the stage for a gravity assist from a flyby of Earth on Oct 9, 2013. Juno will arrive at Jupiter on July 4, 2016.
DIGITAL JUICE
DIGITAL JUICE
Leaders' emotional cues may predict acts of terror or political aggression
Leaders' emotional cues may predict acts of terror or political aggression: Leaders often use rousing speeches to evoke powerful emotions, and those emotions may predict when a group will commit an act of violence or terrorism, according to new research. Analysis of speeches delivered by government, activist and terrorist leaders found that leaders' expressions of anger, contempt and disgust spiked immediately before their group committed an act of violence.
DIGITAL JUICE
DIGITAL JUICE
What's Ambergris? Behind the $60k Whale-Waste Find
What's Ambergris? Behind the $60k Whale-Waste Find:
A beachcombing U.K. boy just stumbled upon a chunk of ambergris possibly worth up to $63,000. So what exactly is it, and why is it worth so much?
DIGITAL JUICE
A beachcombing U.K. boy just stumbled upon a chunk of ambergris possibly worth up to $63,000. So what exactly is it, and why is it worth so much?
DIGITAL JUICE
Does wisdom really come with age? It depends on the culture
Does wisdom really come with age? It depends on the culture: It's certainly comforting to think that aging benefits the mind, if not the body. But do we really get wiser with age? Research suggests that having wisdom means includes being good at resolving conflict, but conflict is not handled the same way across cultures. A new study examines how the resolution of conflict and, by extension, wisdom differ between Japanese and American cultures.
DIGITAL JUICE
DIGITAL JUICE
Lighting and Music Reduce Fast Food Meal Size
Lighting and Music Reduce Fast Food Meal Size:
In lab tests, music and lighting can affect how much people eat. Now a study has found that changing the ambiance of a fast food restaurant to more of a fine-dining atmosphere lessened the amount of food people crammed into their pie holes. To quote the paper, “softening the lighting and music led people to eat less, to rate the food as more enjoyable, and to spend just as much.”
[More]
DIGITAL JUICE
In lab tests, music and lighting can affect how much people eat. Now a study has found that changing the ambiance of a fast food restaurant to more of a fine-dining atmosphere lessened the amount of food people crammed into their pie holes. To quote the paper, “softening the lighting and music led people to eat less, to rate the food as more enjoyable, and to spend just as much.”
[More]
DIGITAL JUICE
Shedding new light on one of diabetes' most dangerous complications
Shedding new light on one of diabetes' most dangerous complications: For many diabetics, monitoring their condition involves much more than adhering to a routine of glucose sensing and insulin injections. It also entails carefully monitoring the ongoing toll this disease takes on their body. An innovative new optical diagnostic tool may soon make it easier to diagnose and monitor one of the most serious complications of diabetes, peripheral arterial disease (PAD).
DIGITAL JUICE
DIGITAL JUICE
The science behind our weirdest behaviours
The science behind our weirdest behaviours: Could sneezing, yawning and tickling have a shared function? PLUS The call to bring back parasites and the woolly credulity of Sheldrake's swansong
DIGITAL JUICE
 |  |
DIGITAL JUICE
First implantation of prototype bionic eye with 24 electrodes: 'All of a sudden I could see a little flash of light'
First implantation of prototype bionic eye with 24 electrodes: 'All of a sudden I could see a little flash of light': In a major development, researchers in Australia have successfully performed the first implantation of an early prototype bionic eye with 24 electrodes. A patient with profound vision loss due to retinitis pigmentosa, an inherited condition, has now received the implant that enables her to experience some vision.
DIGITAL JUICE
DIGITAL JUICE
Climate indices to watch
Climate indices to watch:
What is the most important climate condition to keep tabs on? We have recently mentioned the record-low Arctic sea-ice extent, but hurricanes this year seem to be getting the most attention because of timing ofHurricane Isaac (I know of no evidence suggesting that the Arctic sea-ice has such a direct impact on U.S. politics!).
In addition, the status of ENSO issued by NOAA on August 27, 2012, states that El Niño conditions are likely to develop during August or September 2012, although the present state is classified as ‘ENSO-neutral‘. El Niño has a strong influence on local economies and societies in fairly extensive regions of the world. ENSO is a natural phenomenon, but may change under a changing climate and is interesting to watch over the long term.
It’s important to avoid getting lost into single indicators, however, as the climate system is complex, with many different parts interacting with one another. The American Meteorological Society (AMS) recently put out a statement on climate change, referring to a wide range of different climate indicators (here is a link for the most common ones). The AMS is not alone – the National Academy for Sciences (NAS) is also concerned about our climate and its many aspects: A fairly recent movie called Climate change at the NAS provides a comprehensive overview.
Both AMS and NAS accounts provide a rich picture of many different aspects with many different (important) details, which make them fairly long and complicated. This is why simple indices sometimes are used – to convey a simple message. We need both, and that’s why the NAS video and the AMS statement are so valuable – at least for the readers who understand what they are talking about. I’m not sure that everybody does, though.
DIGITAL JUICE
What is the most important climate condition to keep tabs on? We have recently mentioned the record-low Arctic sea-ice extent, but hurricanes this year seem to be getting the most attention because of timing ofHurricane Isaac (I know of no evidence suggesting that the Arctic sea-ice has such a direct impact on U.S. politics!).
In addition, the status of ENSO issued by NOAA on August 27, 2012, states that El Niño conditions are likely to develop during August or September 2012, although the present state is classified as ‘ENSO-neutral‘. El Niño has a strong influence on local economies and societies in fairly extensive regions of the world. ENSO is a natural phenomenon, but may change under a changing climate and is interesting to watch over the long term.
It’s important to avoid getting lost into single indicators, however, as the climate system is complex, with many different parts interacting with one another. The American Meteorological Society (AMS) recently put out a statement on climate change, referring to a wide range of different climate indicators (here is a link for the most common ones). The AMS is not alone – the National Academy for Sciences (NAS) is also concerned about our climate and its many aspects: A fairly recent movie called Climate change at the NAS provides a comprehensive overview.
Both AMS and NAS accounts provide a rich picture of many different aspects with many different (important) details, which make them fairly long and complicated. This is why simple indices sometimes are used – to convey a simple message. We need both, and that’s why the NAS video and the AMS statement are so valuable – at least for the readers who understand what they are talking about. I’m not sure that everybody does, though.
DIGITAL JUICE
One-Minute Physics: How to travel through the Earth
One-Minute Physics: How to travel through the Earth: Watch an animation that shows what would happen if you fell in a hole through the Earth
DIGITAL JUICE
| |
DIGITAL JUICE
Feedback: God saves the Loch Ness monster
Feedback: God saves the Loch Ness monster: Creationism meets Nessie, the value of sauce, minute-by-minute coverage of a 10-second race, and more (full text available to subscribers)
DIGITAL JUICE
| |
DIGITAL JUICE
New 'Rust-Tracker' to monitor deadly wheat fungus in 27 nations
New 'Rust-Tracker' to monitor deadly wheat fungus in 27 nations: The world's top wheat experts have reported a breakthrough in their ability to track Ug99 and related strains of a deadly and rapidly mutating wheat pathogen called stem rust that threatens wheat fields from East Africa to South Asia. The creators of the "Rust-Tracker" say they now can monitor an unprecedented 42 million hectares of wheat in 27 developing countries.
DIGITAL JUICE
DIGITAL JUICE
T-Mobile Austria Updates Prices Wirelessly
T-Mobile Austria Updates Prices Wirelessly: Thanks to RFID-based electronic shelf displays, the company can change product information across all of its retail stores in an instant.
DIGITAL JUICE
DIGITAL JUICE
Canada's Foreign Affairs Department Uses RFID to Track Assets Worldwide
Canada's Foreign Affairs Department Uses RFID to Track Assets Worldwide: The agency is piloting a solution that employs passive UHF tags and readers to better monitor the furniture and appliances located within the residences of its officials and employees stationed abroad.
DIGITAL JUICE
DIGITAL JUICE
RFID Brings Breast Cancer Awareness to Facebook
RFID Brings Breast Cancer Awareness to Facebook: Participants in Susan G. Komen for the Cure's three-day, 60-mile events wear EPC Gen 2 RFID tags that link to Facebook pages, to update friends and family members about their progress.
DIGITAL JUICE
DIGITAL JUICE
RFID Scores Game Show by the Seat of Contestants' Pants
RFID Scores Game Show by the Seat of Contestants' Pants: Oh Sit! uses RFID tags and readers from RFID Inc. and PTP to automate scoring in an extreme version of musical chairs.
DIGITAL JUICE
DIGITAL JUICE
Industrial Companies Use RFID-enabled Washers to Track Heavy Equipment
Industrial Companies Use RFID-enabled Washers to Track Heavy Equipment: Holland 1916's solution allows a user to simply install a washer onto a bolt on each item, eliminating the need to drill screw holes or weld RFID tags onto an object's exterior.
DIGITAL JUICE
DIGITAL JUICE
RFID News Roundup
RFID News Roundup: Fujitsu upgrades its ultra-rugged UHF RFID tags for commercial laundry systems; OmniAir Certification Services accredits MET Labs as first test lab for tolling interoperability certification; Coast 2 Coast's facilities-management services and solutions to leverage Centrak's RTLS; Chicago bus operator Pace selects Cubic for payment system; Adidas partied with RFID during the Olympics.
DIGITAL JUICE
DIGITAL JUICE
Patterns for research in machine learning
Patterns for research in machine learning:
There are a handful of basic code patterns that I wish I was more aware of when I started research in machine learning. Each on its own may seem pointless, but collectively they go a long way towards making the typical research workflow more efficient. Here they are:
Click here for discussion and examples for each item. Also see Charles Sutton’s and HackerNews’ thoughts on the same topic.
My guess is that these patterns will not only be useful for machine learning, but also any other computational work that involves either a) processing large amounts of data, or b) algorithms that take a significant amount of time to execute. Share this list with your students and colleagues. Trust me, they’ll appreciate it.
DIGITAL JUICE
There are a handful of basic code patterns that I wish I was more aware of when I started research in machine learning. Each on its own may seem pointless, but collectively they go a long way towards making the typical research workflow more efficient. Here they are:
- Separate code from data.
- Separate input data, working data and output data.
- Save everything to disk frequently.
- Separate options from parameters.
- Do not use global variables.
- Record the options used to generate each run of the algorithm.
- Make it easy to sweep options.
- Make it easy to execute only portions of the code.
- Use checkpointing.
- Write demos and tests.
Click here for discussion and examples for each item. Also see Charles Sutton’s and HackerNews’ thoughts on the same topic.
My guess is that these patterns will not only be useful for machine learning, but also any other computational work that involves either a) processing large amounts of data, or b) algorithms that take a significant amount of time to execute. Share this list with your students and colleagues. Trust me, they’ll appreciate it.
DIGITAL JUICE
NYAS ML 2012 and ICML 2013
NYAS ML 2012 and ICML 2013:
The New York Machine Learning Symposium is October 19 with a 2 page abstract deadline due September 13 via email with subject “Machine Learning Poster Submission” sent to physicalscience@nyas.org. Everyone is welcome to submit. Last year’s attendance was 246 and I expect more this year.
The primary experiment for ICML 2013 is multiple paper submission deadlines with rolling review cycles. The key dates are October 1, January 7, and February 15. This is an attempt to shift ICML further towards a journal style review process and reduce peak load. The “not for proceedings” experiment from this year’s ICML is not continuing.
DIGITAL JUICE
The New York Machine Learning Symposium is October 19 with a 2 page abstract deadline due September 13 via email with subject “Machine Learning Poster Submission” sent to physicalscience@nyas.org. Everyone is welcome to submit. Last year’s attendance was 246 and I expect more this year.
The primary experiment for ICML 2013 is multiple paper submission deadlines with rolling review cycles. The key dates are October 1, January 7, and February 15. This is an attempt to shift ICML further towards a journal style review process and reduce peak load. The “not for proceedings” experiment from this year’s ICML is not continuing.
DIGITAL JUICE
Exploring Device Orientation and Motion
Exploring Device Orientation and Motion:
Today, we released a prototype implementation of the W3C DeviceOrientation Event Specification draft on HTML5Labs.com. This specification defines new DOM events that provide information about the physical orientation and motion of a device. Such APIs will let Web developers easily deliver advanced Web user experiences leveraging modern devices' sensors.
With the Device Orientation API, developers can explore new input mechanisms for games, new gestures for apps (such as “shake to clear the screen” or “tilt to zoom”) or even augmented reality experiences. The prototype’s installation includes a sample game to get you started in understanding the API.
Video showing the concepts explained in this post in action
The Device Orientation API exposes two different types of sensor data: orientation and motion.
When the physical orientation of the device is changed (e.g. the user tilts or rotates it), the
When a device is being moved or rotated (more accurately, accelerated), the
You can download the prototype at HTML5Labs. This prototype requires Internet Explorer 10 running on devices with accelerometer sensors supported by Windows 8. The prototype works as an extension to Internet Explorer on the desktop, where developers can get a first-hand look at the APIs. To get started building your own pages with the prototype, all you need to do is install the prototype and then include a reference to the DeviceOrientation.js script file (copied to the desktop after installing the prototype):
We want to hear from developers on this prototype implementation of the W3C Device Orientation Event Specification, so please let us know what you think by commenting on this post or sending us a message.
—Abu Obeida Bakhach, Program Manager, Microsoft Open Technologies Inc.
—Jacob Rossi, Program Manager, Internet Explorer
DIGITAL JUICE
Today, we released a prototype implementation of the W3C DeviceOrientation Event Specification draft on HTML5Labs.com. This specification defines new DOM events that provide information about the physical orientation and motion of a device. Such APIs will let Web developers easily deliver advanced Web user experiences leveraging modern devices' sensors.
How This Helps Developers
With the Device Orientation API, developers can explore new input mechanisms for games, new gestures for apps (such as “shake to clear the screen” or “tilt to zoom”) or even augmented reality experiences. The prototype’s installation includes a sample game to get you started in understanding the API.
Video showing the concepts explained in this post in action
How This Works
The Device Orientation API exposes two different types of sensor data: orientation and motion.
When the physical orientation of the device is changed (e.g. the user tilts or rotates it), the
deviceorientation event is fired at the window and supplies the alpha, beta, and gamma angles of rotation (expressed in degrees):<div id="directions"></div><script>window.addEventListener("deviceorientation", findNorth);function findNorth(evt) {var directions = document.getElementById("directions");if (evt.alpha < 5 || evt.alpha > 355) {directions.innerHTML = "North!";} else if (evt.alpha < 180) {directions.innerHTML = "Turn Left";} else {directions.innerHTML = "Turn Right";}}</script>When a device is being moved or rotated (more accurately, accelerated), the
devicemotion event is fired at the window and provides acceleration (both with and without the effects of gravitational acceleration on the device, expressed in m/s2) in the x, y, and z axis as well as the rate of change in the alpha, beta, and gamma rotation angles (expressed in deg/s):<div id="status"></div><script>window.addEventListener("devicemotion", detectShake);function detectShake(evt) {var status = document.getElementById("status");var accl = evt.acceleration;if (accl.x > 1.5 || accl.y > 1.5 || accl.z > 1.5) {status.innerHTML = "EARTHQUAKE!!!";} else {status.innerHTML = "All systems go!";}}</script>
Trying Out The Prototype
You can download the prototype at HTML5Labs. This prototype requires Internet Explorer 10 running on devices with accelerometer sensors supported by Windows 8. The prototype works as an extension to Internet Explorer on the desktop, where developers can get a first-hand look at the APIs. To get started building your own pages with the prototype, all you need to do is install the prototype and then include a reference to the DeviceOrientation.js script file (copied to the desktop after installing the prototype):
<script type="text/javascript" src="DeviceOrientation.js"></script>
We Want Your Feedback
We want to hear from developers on this prototype implementation of the W3C Device Orientation Event Specification, so please let us know what you think by commenting on this post or sending us a message.
—Abu Obeida Bakhach, Program Manager, Microsoft Open Technologies Inc.
—Jacob Rossi, Program Manager, Internet Explorer
DIGITAL JUICE
MySQL kill could be so much more exciting
MySQL kill could be so much more exciting:
When I kill a query or connection, whoever is running it gets a boring message about what happens. Wouldn’t it be fun and useful to be able to specify the error message the user should see? Imagine the possibilities:
Joking aside, relevant error messages would be great for all involved.
Further Reading:
DIGITAL JUICE
When I kill a query or connection, whoever is running it gets a boring message about what happens. Wouldn’t it be fun and useful to be able to specify the error message the user should see? Imagine the possibilities:
mysql> KILL 10282, "Sorry, no cigar today. Try again tomorrow."
Joking aside, relevant error messages would be great for all involved.
Further Reading:
- An exciting week for InnoDB and MySQL
- MySQL’s ERROR 1025 explained
- The unexpected consequences of SELinux
- How to notify event listeners in MySQL
- Killing idle transactions to increase database uptime
DIGITAL JUICE
Debugging metadata locking in MySQL 5.5
Debugging metadata locking in MySQL 5.5:
MySQL 5.1 added a long-needed feature: INFORMATION_SCHEMA tables for inspecting locks held and waited-for inside of InnoDB. With this addition, it became possible to figure out who is blocking whom.
MySQL 5.5 changed a lot more things inside the server. One of the new features is improved metadata locking. This solves a lot of inconsistencies and bugs that were previously possible. The output of
Unfortunately, the additional locking is not possible for the DBA to inspect. Where InnoDB’s storage-engine-level locking used to be opaque and was then made transparent, now server-level locking is a mystery, and in my experience much more likely to become a problem than it used to be. (The server now does a lot more locking of things it previously neglected to lock.)
I have recently run into a number of circumstances where metadata locking prevents users from doing any work. The locking is not inside InnoDB, so the INFORMATION_SCHEMA tables show nothing at all. I tried using
It feels like a bug in metadata locking to me, but I will need to dig in more deeply the next time I see it happen. Unfortunately, it’s unlikely that I’ll be able to create a reproducible test case, because there’s no way to actually see what is happening. I hope that a future version of MySQL will include a more comprehensive set of tables for inspecting locks, requests, and waits at all layers of the server.
Further Reading:
DIGITAL JUICE
MySQL 5.1 added a long-needed feature: INFORMATION_SCHEMA tables for inspecting locks held and waited-for inside of InnoDB. With this addition, it became possible to figure out who is blocking whom.
MySQL 5.5 changed a lot more things inside the server. One of the new features is improved metadata locking. This solves a lot of inconsistencies and bugs that were previously possible. The output of
SHOW PROCESSLIST is also changed; instead of the venerable Locked status, there are more fine-grained status indicators such as Waiting for table metadata lock.Unfortunately, the additional locking is not possible for the DBA to inspect. Where InnoDB’s storage-engine-level locking used to be opaque and was then made transparent, now server-level locking is a mystery, and in my experience much more likely to become a problem than it used to be. (The server now does a lot more locking of things it previously neglected to lock.)
I have recently run into a number of circumstances where metadata locking prevents users from doing any work. The locking is not inside InnoDB, so the INFORMATION_SCHEMA tables show nothing at all. I tried using
mysqladmin debug and looking at the contents of the server’s error log, but the table in question was shown as unused in the output there. I tried examining each of the other connections to the server, but was unable to find any connection that I believed could have touched the table and locked it. The lock wait was ongoing for a long time, and no other transaction had been started previous to that, so it did not seem that a long-running transaction could have touched the table and was keeping a metadata lock open.It feels like a bug in metadata locking to me, but I will need to dig in more deeply the next time I see it happen. Unfortunately, it’s unlikely that I’ll be able to create a reproducible test case, because there’s no way to actually see what is happening. I hope that a future version of MySQL will include a more comprehensive set of tables for inspecting locks, requests, and waits at all layers of the server.
Further Reading:
- How to find out who is locking a table in MySQL
- How to give locking hints in MySQL
- How to capture debugging information with mk-loadavg
- How I patched InnoDB to show locks held
- How to monitor InnoDB lock waits
DIGITAL JUICE
the Grand Mosque, Muscat, Oman
the Grand Mosque, Muscat, Oman: 
I did think about asking if any of you could identify where this was taken but thought it was probably just a bit too obscure: it's the ceiling, just inside the doorway of the Grand Mosque in Muscat, Oman.
And in case you're wondering, it was processed with one curve and a very minor boost to the saturation using the hue/sat tool.
DIGITAL JUICE
I did think about asking if any of you could identify where this was taken but thought it was probably just a bit too obscure: it's the ceiling, just inside the doorway of the Grand Mosque in Muscat, Oman.
And in case you're wondering, it was processed with one curve and a very minor boost to the saturation using the hue/sat tool.
captured camera lens focal length aperture shutter speed shooting mode exposure bias metering mode ISO flash image quality RAW converter image editor plugins (etc) cropped? | 10.06am on 14/3/12 Canon 5D Mark II EF 70-200mm f/2.8L IS USM 94mm f/4.0 1/25 aperture priority -0.7 evaluative 800 no RAW Camera Raw Photoshop CS5  none minor transformation |
DIGITAL JUICE
Faces and Places, Oman #13
Faces and Places, Oman #13: 
Since starting work on my (relatively) new Udemy course on black and white photography pretty much all of my black and white conversions have been intentional, i.e. I've started out knowing that I was going to produce a black and white image. With this image though I did try to produce a colour version but just couldn't seem to get it to work. Maybe I've just been spending too long looking at black and white images, or maybe this is another example of an image that just doesn't have the same impact in colour. I don't know. I am happy with how it turned out though.
And on a related note: today's the last day do grab a $20 early bird discount on my The Art of Black and White Photography course. If you haven't seen it yet: we launched it a month ago today and now have 408 subscribers and 17 five star reviews. A HUGE thank you to everyone who's signed up so far :-)
DIGITAL JUICE
Since starting work on my (relatively) new Udemy course on black and white photography pretty much all of my black and white conversions have been intentional, i.e. I've started out knowing that I was going to produce a black and white image. With this image though I did try to produce a colour version but just couldn't seem to get it to work. Maybe I've just been spending too long looking at black and white images, or maybe this is another example of an image that just doesn't have the same impact in colour. I don't know. I am happy with how it turned out though.
And on a related note: today's the last day do grab a $20 early bird discount on my The Art of Black and White Photography course. If you haven't seen it yet: we launched it a month ago today and now have 408 subscribers and 17 five star reviews. A HUGE thank you to everyone who's signed up so far :-)
captured camera lens focal length aperture shutter speed shooting mode exposure bias metering mode ISO flash image quality RAW converter image editor plugins (etc) cropped? | 9.38am on 16/3/12 Canon 5D Mark II EF 70-200mm f/2.8L IS USM 195mm f/5.6 1/80 aperture priority +0.7 evaluative 200 no RAW Camera Raw Photoshop CS6 none minor |
DIGITAL JUICE
Surprises
Surprises:
I’ve given examples in the past of how you can be suprised by Oracle when a mechanism that has “always worked” in the past suddenly just doesn’t work because some unexpected internal coding mechanism hits a boundary condition. One example I gave of this was rebuilding indexes online – where a limitation on the key size of index organized tables made it impossible to do an online rebuild of an index on a heap table because of an ORA-01450 (maximum key length) error that was raised against the (index-organized) journal table that Oracle creates internally to support the rebuild.
Slightly more recently the topic of the sys_op_map_nonnull() function came up in some comments about the ora_hash function – and I made a passing comment about the function appearing in materialized view refreshes and then never followed it up. Fortunately I got an email from Jared Still and Don Seiler a few days ago which contained a nice short demonstration of the problem so, with their permission, I’ve published it below.
We start by creating a table with a varchar2(4000) column – and in my case I can do this because my database character set is a single-byte character set – then build a materialized view log on the table:
You’ll notice that the code creates a non-unique index to support the primary key constraint. I haven’t checked to see if this makes any difference to the outcome of the demonstration, but anyone who wants to repeat the test can do this.
After creating the log, the next piece of code creates an aggregate materialized view on a pre-built table.
The primary key constraint on the materialized view is also protected by a non-unique index but in this case this is a requirement of Oracle’s implementation – if you support the primary (or unique) constraint with a unique index then a fast refresh may, in general, end up generating a “duplicate key in index” error.
Finally we insert a row into the base table and call for a fast refresh:
You’ll note that in my insert statement I’ve used a substitution variable ( &m_length ) to dictate the length of the string I insert for column data. If I set this to 4,000 the refresh fails; any smaller value and the refresh succeeds. (Note: the line numbers reported in the ORA-06512 errors will vary with version of Oracle, but the ORA-01706 error is always the same.)
You can find out what’s gone wrong when you re-run the refresh with sql_trace enabled – in the trace file you’ll find a statement like the following (which I’ve tidied up considerably):
Spot the problem – it’s the use of the sys_op_map_nonnull() function. As it says in the comments I mentioned above, this function adds one byte to the existing value – so if the input is already 4,000 bytes long the output exceeds the legal limit and results in a ORA-01706. (Try to select sys_op_map_nonnull(rpad(‘x’,4000,’x')) from dual if you want a simpler demonstration.)
The worrying thing about this particular example is that you could run a production system with these definitions and everything could work perfectly for ages, until the day that someone inserted data that used the full 4,000 byte length of the column declaration – and then your materialized view can’t be refreshed.
And here’s an annoying detail that applies in this case – I altered the data column to declare it as not null, which means the sys_op_map_nonnull() call is redundant anyway. That’s the drawback, I guess, of reusable code – sometimes you re-use it when perhaps you ought to “special case” it.
Footnote: it is possible that Oracle has some code that decides whether or not to use the merge command to do the refresh – it’s possible, though I haven’t checked it, that Oracle could choose between merge and a “delete/insert” cycle; if you find that Oracle doesn’t crash when you try this test with the 4,000 byte character string then check the trace file to see whether Oracle has used a different strategy for the refresh.
Teaser: for an even stranger behaviour relating to fast refresh materialized views, see next weeks exciting episode.
DIGITAL JUICE
I’ve given examples in the past of how you can be suprised by Oracle when a mechanism that has “always worked” in the past suddenly just doesn’t work because some unexpected internal coding mechanism hits a boundary condition. One example I gave of this was rebuilding indexes online – where a limitation on the key size of index organized tables made it impossible to do an online rebuild of an index on a heap table because of an ORA-01450 (maximum key length) error that was raised against the (index-organized) journal table that Oracle creates internally to support the rebuild.
Slightly more recently the topic of the sys_op_map_nonnull() function came up in some comments about the ora_hash function – and I made a passing comment about the function appearing in materialized view refreshes and then never followed it up. Fortunately I got an email from Jared Still and Don Seiler a few days ago which contained a nice short demonstration of the problem so, with their permission, I’ve published it below.
We start by creating a table with a varchar2(4000) column – and in my case I can do this because my database character set is a single-byte character set – then build a materialized view log on the table:
create table frbug
as
select
t.owner,
t.table_name,
cast('x' as varchar2(4000)) data
from
all_tables t
where
rownum <= 2000
;
alter table frbug modify data not null;
create index frbug_pk_idx on frbug(owner,table_name);
alter table frbug add constraint frbug_pk primary key (owner,table_name);
create materialized view log on frbug
with primary key, sequence, rowid
(data)
including new values
;
You’ll notice that the code creates a non-unique index to support the primary key constraint. I haven’t checked to see if this makes any difference to the outcome of the demonstration, but anyone who wants to repeat the test can do this.
After creating the log, the next piece of code creates an aggregate materialized view on a pre-built table.
create table frbug_mv
as
select
owner,
table_name,
data,
1 mvcount
from
frbug
;
create index frbug_mv_pk_idx on frbug_mv(owner,table_name);
alter table frbug_mv add constraint frbug_mv_pk primary key (owner,table_name);
create materialized view frbug_mv
on prebuilt table
refresh fast
enable query rewrite
as
select
owner, table_name, data, count(*) mvcount
from
frbug
group by
owner, table_name, data
;
The primary key constraint on the materialized view is also protected by a non-unique index but in this case this is a requirement of Oracle’s implementation – if you support the primary (or unique) constraint with a unique index then a fast refresh may, in general, end up generating a “duplicate key in index” error.
Finally we insert a row into the base table and call for a fast refresh:
insert into frbug(owner, table_name, data)
values('JKSTILL','BIG_TABLE',rpad('is this too big?', &m_length,'X'))
;
commit;
exec dbms_mview.refresh('FRBUG_MV', method => 'F')
*
ERROR at line 1:
ORA-12008: error in materialized view refresh path
ORA-01706: user function result value was too large
ORA-06512: at "SYS.DBMS_SNAPSHOT", line 2254
ORA-06512: at "SYS.DBMS_SNAPSHOT", line 2460
ORA-06512: at "SYS.DBMS_SNAPSHOT", line 2429
ORA-06512: at line 1
You’ll note that in my insert statement I’ve used a substitution variable ( &m_length ) to dictate the length of the string I insert for column data. If I set this to 4,000 the refresh fails; any smaller value and the refresh succeeds. (Note: the line numbers reported in the ORA-06512 errors will vary with version of Oracle, but the ORA-01706 error is always the same.)
You can find out what’s gone wrong when you re-run the refresh with sql_trace enabled – in the trace file you’ll find a statement like the following (which I’ve tidied up considerably):
MERGE INTO TEST_USER.FRBUG_MV SNA$
USING (
SELECT /*+ OPT_ESTIMATE(QUERY_BLOCK MAX=1000) */
DLT$0.OWNER GB0, DLT$0.TABLE_NAME GB1, DLT$0.DATA GB2, SUM(1) D0
FROM (
SELECT
CHARTOROWID(MAS$.M_ROW$) RID$, MAS$.DATA, MAS$.TABLE_NAME, MAS$.OWNER
FROM
TEST_USER.MLOG$_FRBUG MAS$
WHERE
MAS$.SNAPTIME$ > :1
) AS OF SNAPSHOT (:2) DLT$0
GROUP BY
DLT$0.OWNER, DLT$0.TABLE_NAME, DLT$0.DATA
)AV$
ON
(
SYS_OP_MAP_NONNULL(SNA$.OWNER) = SYS_OP_MAP_NONNULL(AV$.GB0)
AND SYS_OP_MAP_NONNULL(SNA$.TABLE_NAME) = SYS_OP_MAP_NONNULL(AV$.GB1)
AND SYS_OP_MAP_NONNULL(SNA$.DATA) = SYS_OP_MAP_NONNULL(AV$.GB2)
)
WHEN MATCHED THEN
UPDATE SET SNA$.MVCOUNT = SNA$.MVCOUNT+AV$.D0
WHEN NOT MATCHED THEN
INSERT (SNA$.OWNER, SNA$.TABLE_NAME, SNA$.DATA,SNA$.MVCOUNT)
VALUES (AV$.GB0, AV$.GB1, AV$.GB2, AV$.D0)
;
Spot the problem – it’s the use of the sys_op_map_nonnull() function. As it says in the comments I mentioned above, this function adds one byte to the existing value – so if the input is already 4,000 bytes long the output exceeds the legal limit and results in a ORA-01706. (Try to select sys_op_map_nonnull(rpad(‘x’,4000,’x')) from dual if you want a simpler demonstration.)
The worrying thing about this particular example is that you could run a production system with these definitions and everything could work perfectly for ages, until the day that someone inserted data that used the full 4,000 byte length of the column declaration – and then your materialized view can’t be refreshed.
And here’s an annoying detail that applies in this case – I altered the data column to declare it as not null, which means the sys_op_map_nonnull() call is redundant anyway. That’s the drawback, I guess, of reusable code – sometimes you re-use it when perhaps you ought to “special case” it.
Footnote: it is possible that Oracle has some code that decides whether or not to use the merge command to do the refresh – it’s possible, though I haven’t checked it, that Oracle could choose between merge and a “delete/insert” cycle; if you find that Oracle doesn’t crash when you try this test with the 4,000 byte character string then check the trace file to see whether Oracle has used a different strategy for the refresh.
Teaser: for an even stranger behaviour relating to fast refresh materialized views, see next weeks exciting episode.
DIGITAL JUICE
Nokia Lumia 820 Photo Leaked?
Nokia Lumia 820 Photo Leaked?:
Here’s the Nokia Lumia 820 according to @evleaks. Not other description other than it has a 4.3-inch display. The new colors are refreshing. I like that red one. Check it out
We’ll have to wait for the September 5th press conference to see if this device is for real.
There’s also a webcast for those who can’t attend in person. Just go to www.nokia.com/webcast on the 5th September at 10:00am (EDT).
What do you think?
DIGITAL JUICE
Here’s the Nokia Lumia 820 according to @evleaks. Not other description other than it has a 4.3-inch display. The new colors are refreshing. I like that red one. Check it out
We’ll have to wait for the September 5th press conference to see if this device is for real.
There’s also a webcast for those who can’t attend in person. Just go to www.nokia.com/webcast on the 5th September at 10:00am (EDT).
What do you think?
If you enjoyed this article, you might also like...
- Nokia Lumia 601 Images Leaked, First Windows Phone with EDoF Camera?
- Nokia Lumia 920 Leaked?
- Nokia Phi Design Patent Revealed, Looks Like a Brick
DIGITAL JUICE
Nokia Lumia 920 Leaked?
Nokia Lumia 920 Leaked?:
A photo of the Nokia Lumia 920 was just leaked by @evleaks. They added that it has a 4.5-inch display with PureView technology. There’s no sign of Xenon flash or the number of megapixels for the camera. It also looks a little thick. Check it out.
We’ll have to wait for the September 5th press conference to see if this device is for real.
There’s also a webcast for those who can’t attend in person. Just go to www.nokia.com/webcast on the 5th September at 10:00am (EDT).
What do you think?
DIGITAL JUICE
A photo of the Nokia Lumia 920 was just leaked by @evleaks. They added that it has a 4.5-inch display with PureView technology. There’s no sign of Xenon flash or the number of megapixels for the camera. It also looks a little thick. Check it out.
We’ll have to wait for the September 5th press conference to see if this device is for real.
There’s also a webcast for those who can’t attend in person. Just go to www.nokia.com/webcast on the 5th September at 10:00am (EDT).
What do you think?
If you enjoyed this article, you might also like...
- Nokia Lumia 820 Photo Leaked?
- Nokia Lumia Coming to Verizon This Year
- Nokia’s Chris Weber Calls Out Samsung on Twitter
DIGITAL JUICE
Subscribe to:
Posts (Atom)